Cross-species modeling of plant genomes at single nucleotide resolution using a pre-trained DNA language model.
JingJing ZhaiAaron GokaslanYair SchiffAna BerthelZong-Yan LiuZachary R MillerArmin SchebenMichelle C StitzerMaria Cinta RomayEdward S BucklerVolodymyr KuleshovPublished in: bioRxiv : the preprint server for biology (2024)
Understanding the function and fitness effects of diverse plant genomes requires transferable models. Language models (LMs) pre-trained on large-scale biological sequences can learn evolutionary conservation, thus expected to offer better cross-species prediction through fine-tuning on limited labeled data compared to supervised deep learning models. We introduce PlantCaduceus, a plant DNA LM based on the Caduceus and Mamba architectures, pre-trained on a carefully curated dataset consisting of 16 diverse Angiosperm genomes. Fine-tuning PlantCaduceus on limited labeled Arabidopsis data for four tasks involving transcription and translation modeling demonstrated high transferability to maize that diverged 160 million years ago, outperforming the best baseline model by 1.45-fold to 7.23-fold. PlantCaduceus also enables genome-wide deleterious mutation identification without multiple sequence alignment (MSA). PlantCaduceus demonstrated a threefold enrichment of rare alleles in prioritized deleterious mutations compared to MSA-based methods and matched state-of-the-art protein LMs. PlantCaduceus is a versatile pre-trained DNA LM expected to accelerate plant genomics and crop breeding applications.
Keyphrases
- genome wide
- resistance training
- circulating tumor
- single molecule
- cell free
- deep learning
- transcription factor
- cell wall
- machine learning
- air pollution
- body composition
- electronic health record
- dna methylation
- autism spectrum disorder
- pet imaging
- big data
- nucleic acid
- computed tomography
- small molecule
- working memory
- plant growth