The two most common questions I get via cold email are 1) what should I work on, and 2) how do I get a job doing research, engineering, etc.?
I wrote a new post summarizing my advice for people trying to enter Biology+ML, from undergrads to early career.
�� (link below)
Reinforcement Learning (RL) is quickly becoming the most important skill for AI researchers. Here are the best resources for learning RL for LLMs…
TL;DR: RL is more important now than it has ever been, but (probably due to its complexity) there aren’t a ton of great resources for learning it online. I’ve been doing a lot of reading / learning on RL recently, so I wanted to share the best resources I’ve found. Links to all resources are provided in the image below.
(1) RLHF book. Nathan is a long-time RL researcher and an expert on LLM alignment / post-training. He decided to write an entire book on (LLM-focused) RL techniques and has been slowly expanding / iterating on the book over the last year. This is the most comprehensive RL resource that is currently available, and it’s an especially great resource for those who are unfamiliar with RL and still need to learn the basics.
(2) The Spinning up with Deep RL Course from OpenAI–despite being created in ~2018–has stood the test of time and is one of the best tutorials for learning RL. This course builds up to understanding PPO, which is one of the most widely used algorithms for RL with LLMs. Plus, understanding related algorithms (policy gradients, TRPO, etc.) will help a lot with gaining an understanding of new RL algorithms like GRPO.
(3) PPO / GRPO blog. Jimmy Shi (DeepMind) recently wrote a great blog explaining both PPO (RL algo traditionally used for RLHF) and GRPO (RL algo used for reasoning models). This blog is great and it’s written in a way that is understandable for non-RL people.
(4) HuggingFace RL. HuggingFace has also published numerous useful blogs on the topic of RL. Most recently, they published a blog that explains GRPO and PPO from the ground up (i.e., not assuming any background knowledge on RL). These blogs are inspired by the recent initiative from HuggingFace to create a fully open replication of DeepSeek-R1.
Elucidating the Design Space of Multimodal Protein Language Models
1. This paper presents a systematic exploration of the design space for multimodal protein language models (PLMs), identifying key limitations in token-based structural modeling and proposing architectural and training innovations to overcome them.
2. Tokenizing 3D protein structures into discrete tokens, a common practice in multimodal PLMs like DPLM-2 and ESM3, causes fidelity loss in structural detail. The authors show that bit-wise token representations and residual modeling can recover this loss and improve folding accuracy.
3. Bit-wise supervision significantly outperforms index-based structural tokens, increasing structure prediction accuracy while reducing deviation from ground truth. On PDB test sets, it reduces RMSD from 5.52 to 3.22 and increases TMscore to over 0.90.
4. The authors introduce RESDIFF, a lightweight residual diffusion module to recover fine-grained geometric details lost during tokenization. When combined with bit-based modeling, it further improves folding accuracy with minimal computational overhead.
5. A hybrid generative model is proposed, combining structure encoder, PLM, and decoder as a structure denoising pipeline. This enables direct data-space structure generation using flow matching, improving atomic-level predictions without sacrificing scalability.
6. To incorporate geometric inductive biases, the authors introduce GeoDPLM, which integrates structure-aware attention and pairwise representations, akin to AlphaFold’s PairFormer. These modules refine folding and improve generation diversity.
7. REPA (Representation Alignment to Folding Models) transfers structural semantics from pretrained folding models (e.g., ESMFold) to the PLM. This soft alignment boosts structural diversity and generative performance across architectures.
8. Combining bit-based modeling, geometric modules, and REPA achieves state-of-the-art folding accuracy with only 650M parameters, matching or surpassing 3B-size baselines like ESMFold and MultiFlow, while improving training speed and sample diversity.
9. The study also explores the impact of multimeric protein data. Incorporating multimer chains and position offsets into modeling improves performance for both multimer and monomer structures, indicating shared structural principles.
10. The final recommended setting—GeoDPLM with bit-based modeling—offers the best trade-off between accuracy, diversity, training efficiency, and unconditional generation, laying the foundation for future scalable and general-purpose protein models.
💻Code: https://t.co/LBB0m3PCV4
📜Paper: https://t.co/g2mFd8Bvyz
#proteindesign #bioinformatics #proteinlanguage #multimodalmodels #deeplearning #proteinstructure #generativemodels #AlphaFold #AI4Science #diffusionmodels
ProtFlow: Fast Protein Sequence Design via Flow Matching on Compressed Protein Language Model Embeddings
1. ProtFlow introduces the first flow matching-based generative model for protein sequence design, enabling ultra-fast generation of high-quality sequences from a semantically meaningful latent space derived from protein language models.
2. Unlike traditional diffusion-based methods, ProtFlow leverages rectified flow and reflow to drastically reduce generation steps. With reflow, it achieves high-quality one-step generation—cutting inference time by up to 25–100x compared to leading diffusion models.
3. A major innovation of ProtFlow lies in its redesigned latent space. By compressing high-dimensional protein language model embeddings by up to 16-fold and applying smoothing techniques, it achieves better distribution modeling while remaining computationally efficient.
4. ProtFlow supports joint design of multichain proteins, such as antibodies, by integrating multiple embeddings in a unified flow. This allows coherent generation of structurally interdependent chains, significantly improving realism and applicability.
5. Across general protein and peptide generation tasks, ProtFlow achieves state-of-the-art performance in both sequence quality and foldability. It outperforms leading baselines including DiMA, EvoDiff, and Dirichlet FM on perplexity, structure scores, and distributional similarity.
6. In antimicrobial peptide (AMP) design, ProtFlow generates highly diverse and functionally relevant sequences. It achieves the best AMP classifier scores (Pamp and Pmic), the highest diversity (entropy), and the strongest motif learning (6-mer Jaccard) among all models tested.
7. ProtFlow demonstrates robustness across protein sizes and types. It excels in short peptide design, long-chain proteins, and antibody generation—showing consistent gains in structural confidence (pLDDT), naturalness (TM-score), and sequence diversity.
8. Even with small backbone models (e.g., ESM-2 8M), ProtFlow matches or exceeds large-scale diffusion methods in quality while operating with drastically fewer steps, making it ideal for low-resource or real-time protein design applications.
9. As a modular framework, ProtFlow includes components like encoder-decoder pairs, compressor-decompressor modules, and a BERT-style flow holder. Its flexible architecture supports future scaling, conditional generation, and structure-aware extensions.
📜Paper: https://t.co/0nfU8GezEF
#proteindesign #bioinformatics #computationalbiology #generativemodels #flowmatching #proteinengineering #antibodydesign #AI4Science #diffusionmodels #machinelearning
Applying computational protein design to therapeutic antibody discovery - current state and perspectives
1/ This review explores the role of computational protein design (CPD) in antibody discovery, emphasizing recent advances in machine learning and structural bioinformatics that enable in silico antibody engineering.
2/ Antibodies, the largest class of biotherapeutics, have traditionally been developed through immunization and display technologies. While effective, these methods are time-consuming and depend on host immune responses.
3/ Computational antibody design leverages protein modeling, molecular dynamics, and AI-driven generative approaches to accelerate discovery, improve developability, and optimize binding affinity.
4/ Early computational design methods relied on homology modeling and de novo protein engineering, but recent breakthroughs such as AlphaFold2, ProteinMPNN, and RFDiffusion have revolutionized sequence and structure prediction.
5/ Generative protein design models, including language models like ESM, diffusion models like RFDiffusion, and inverse folding methods like ProteinMPNN, now enable the creation of novel antibody sequences and structures.
6/ AI-powered tools such as AbMPNN and AntiFold have improved the inverse folding of antibodies, achieving high sequence recovery rates and optimizing CDR sequences for better stability and binding.
7/ Advances in co-folding and docking techniques, including AlphaFold-multimer and diffusion-based antibody design, are improving the prediction of antibody-antigen interactions, although challenges remain in achieving clinical-level accuracy.
8/ Developability optimization, a key hurdle in therapeutic antibody design, now benefits from machine learning models that assess stability, immunogenicity, solubility, and manufacturability before clinical testing.
9/ The field is shifting toward multi-property optimization (MPO), integrating binding affinity and developability constraints into generative models to produce antibodies with enhanced clinical potential.
10/ Experimental validation remains a major bottleneck, with limited structural and affinity datasets restricting the scalability of AI-driven antibody design. Large-scale data curation efforts are needed to bridge this gap.
11/ Despite challenges, computational antibody design is rapidly advancing, with AI-driven approaches poised to transform the biopharmaceutical industry by enabling faster and more efficient therapeutic development.
@jadolfbr
📜Paper: https://t.co/mB3B1xOpyA
#ComputationalBiology #AI #MachineLearning #AntibodyDesign #ProteinEngineering #DrugDiscovery #Bioinformatics
Deep Learning for Predicting Biomolecular Binding Sites of Proteins @SPJournals
1/ This study explores recent advances in deep learning models for predicting protein-biomolecule binding sites, a crucial task for drug discovery, mutation analysis, and molecular biology. The work compares sequence-based and structure-based approaches, highlighting their advantages and limitations.
2/ Sequence-based methods leverage amino acid sequences and evolutionary features for fast and efficient predictions. These models are computationally lightweight but struggle to capture spatial features crucial for accurate binding site identification.
3/ Structure-based methods rely on 3D protein structures, incorporating spatial relationships for higher precision. However, they require high-quality structural data, which can be challenging to obtain experimentally or computationally.
4/ The study highlights hybrid models that integrate sequence and structural data, improving accuracy and generalizability. Geometric deep learning, graph neural networks (GNNs), and transformer-based approaches are particularly promising for capturing both local and global molecular features.
5/ Point cloud models and surface property-based methods are emerging as effective ways to model protein binding interfaces. These techniques analyze molecular surfaces for features like hydrophobicity and electrostatics, aiding in accurate binding site prediction.
6/ Multi-task learning frameworks, such as DeepDISOBind, demonstrate the power of capturing shared features across different biomolecular interactions, including DNA, RNA, and protein binding sites. Ensemble learning methods further improve model robustness.
7/ The study identifies key challenges, such as the need for more computationally efficient models that can incorporate dynamic protein conformations. Future advancements may involve integrating molecular dynamics simulations and real-time binding predictions.
8/ Ultimately, the research calls for developing flexible, multimodal AI models that integrate sequence, structure, and physicochemical properties to enhance binding site prediction and expand applications across biomedicine.
📜Paper: https://t.co/ziqeq7FYMO
#DeepLearning #ProteinBinding #Bioinformatics #DrugDiscovery #MachineLearning #ComputationalBiology #AI
Integrating protein sequence embeddings with structure via graph-based deep learning for the prediction of single-residue properties
1/ INFUSSE is a deep learning framework that integrates protein sequence embeddings from large language models (LLMs) with graph-based structural representations to predict single-residue properties, offering a more comprehensive approach to protein modeling.
2/ The framework first fine-tunes sequence embeddings generated by ProtBERT, a bidirectional transformer trained on over 200 million protein sequences, to capture biologically relevant sequence information.
3/ INFUSSE then combines these enriched sequence representations with a geometric graph Laplacian using a diffusive Graph Convolutional Network (diff-GCN), refining predictions by incorporating structural information.
4/ The study applies INFUSSE to predict local residue flexibility (B-factors) in antibody-antigen complexes, demonstrating improved performance over existing machine learning methods, particularly for highly flexible, disordered regions and protein-protein interaction sites.
5/ Compared to models based solely on sequence embeddings, INFUSSE shows a notable increase in predictive accuracy, achieving an average Pearson correlation coefficient (PCC) of 0.71 on antibody-antigen complexes, surpassing state-of-the-art LSTM-based models.
6/ By systematically comparing predictions with and without structural information, INFUSSE reveals how sequence and structure contribute to residue-level properties, highlighting cases where geometric graphs significantly enhance predictive power.
7/ The study finds that structural information is especially valuable for predicting flexibility in complementarity-determining regions (CDRs) of antibodies, epitope-paratope interaction sites, and positions with high sequence variability.
8/ With its ability to integrate sequence and structure in a unified framework, INFUSSE provides a valuable tool for studying protein dynamics, antibody engineering, and broader applications in protein design and bioinformatics.
💻Code: https://t.co/I0NSs1CQAB
📜Paper: https://t.co/hmN7e9PoTy
#ProteinStructure #DeepLearning #Bioinformatics #MachineLearning #GraphNeuralNetworks #ComputationalBiology #AntibodyEngineering #ProteinFlexibility
Deep Learning for Predicting Biomolecular Binding Sites of Proteins @SPJournals
1/ This study explores recent advances in deep learning models for predicting protein-biomolecule binding sites, a crucial task for drug discovery, mutation analysis, and molecular biology. The work compares sequence-based and structure-based approaches, highlighting their advantages and limitations.
2/ Sequence-based methods leverage amino acid sequences and evolutionary features for fast and efficient predictions. These models are computationally lightweight but struggle to capture spatial features crucial for accurate binding site identification.
3/ Structure-based methods rely on 3D protein structures, incorporating spatial relationships for higher precision. However, they require high-quality structural data, which can be challenging to obtain experimentally or computationally.
4/ The study highlights hybrid models that integrate sequence and structural data, improving accuracy and generalizability. Geometric deep learning, graph neural networks (GNNs), and transformer-based approaches are particularly promising for capturing both local and global molecular features.
5/ Point cloud models and surface property-based methods are emerging as effective ways to model protein binding interfaces. These techniques analyze molecular surfaces for features like hydrophobicity and electrostatics, aiding in accurate binding site prediction.
6/ Multi-task learning frameworks, such as DeepDISOBind, demonstrate the power of capturing shared features across different biomolecular interactions, including DNA, RNA, and protein binding sites. Ensemble learning methods further improve model robustness.
7/ The study identifies key challenges, such as the need for more computationally efficient models that can incorporate dynamic protein conformations. Future advancements may involve integrating molecular dynamics simulations and real-time binding predictions.
8/ Ultimately, the research calls for developing flexible, multimodal AI models that integrate sequence, structure, and physicochemical properties to enhance binding site prediction and expand applications across biomedicine.
📜Paper: https://t.co/ziqeq7FYMO
#DeepLearning #ProteinBinding #Bioinformatics #DrugDiscovery #MachineLearning #ComputationalBiology #AI
I’m excited to share our significantly-updated preprint on de novo antibody design, where we now demonstrate the structurally accurate design of scFvs (in addition to VHHs) with RFdiffusion! https://t.co/WdYKu0s1Uc
Genome Modeling and Design Across All Domains of Life with Evo 2
1. Evo 2 is a revolutionary biological foundation model, trained on a massive 9.3 trillion DNA base pairs spanning all domains of life. It can predict and generate genomic sequences at an unprecedented scale, achieving robust performance in functional predictions and sequence design.
2. With a focus on generalist capabilities, Evo 2 operates without task-specific fine-tuning, allowing it to predict mutational effects, from noncoding mutations to clinical variants like BRCA1, with remarkable accuracy across prokaryotic, eukaryotic, and viral genomes.
3. The model can predict mutational effects on protein, RNA, and organismal fitness, both for coding and noncoding variants, providing valuable insights for pathogenicity, splice variants, and functional consequences across diverse species.
4. In an exciting leap forward, Evo 2 can generate whole genomes, including mitochondrial, prokaryotic, and eukaryotic sequences, showcasing its power in creating biologically relevant sequences with naturalness and coherence. It can even generate full chromosomes and epigenomic patterns.
5. The study also highlights Evo 2's use in designing complex biological architectures through inference-time search, where the model’s ability to scale computation allows the generation of sequences with specific chromatin accessibility profiles, enabling "generative epigenomics."
6. Mechanistic interpretability tools used to analyze Evo 2 revealed that it autonomously learns key biological features such as exon-intron boundaries, transcription factor motifs, and protein structures, as well as evolutionary signatures like prophage regions.
7. Evo 2’s release as an open-source model, along with its training and inference code, promises to accelerate biological research and genomic design by providing researchers with the tools needed to explore and manipulate the vast complexity of genomic data.
💻Code: https://t.co/zNSa8dt4qo
📜Paper: https://t.co/yddunAIUb9
#AI #Bioinformatics #GenomeDesign #Genomics #MachineLearning #Bioengineering #SyntheticBiology #OpenScience #Evo2 #ComputationalBiology
Accelerating Antibody Development: Sequence and Structure-Based Models for Predicting Developability Properties through Size Exclusion Chromatography
1/ The study introduces a range of in silico models aimed at accelerating the prediction of antibody developability properties, specifically targeting properties measured via size exclusion chromatography (SEC), like monomer content and retention time differences.
2/ By utilizing protein language models (PLMs), the paper explores how sequence-based predictions can efficiently replace the need for computationally intensive feature extraction methods that depend on structural information, achieving faster screening results.
3/ The research integrates graph neural networks (GNNs) into the prediction pipeline, taking advantage of the structural data of antibodies. This approach, combined with PLMs, further enhances prediction accuracy, particularly for retention time differences in SEC assays.
4/ The paper compares multiple prediction pipelines, including sequence and structure-based features, PLM-only models, GNN, and a hybrid PLM+GNN approach, offering insights into which combination yields the best performance for different antibody properties.
5/ A key finding is that PLM-based pipelines, particularly those using pre-trained models like ESM-2, offer a balance of speed and accuracy, making them promising for high-throughput antibody screening in early-stage drug development.
6/ The study emphasizes the potential of these models to reduce the reliance on expensive, time-consuming experimental assays by leveraging predictive models that can be applied in silico for screening large antibody datasets.
📜Paper: https://t.co/gENGHLkQSH
#AntibodyDevelopment #MachineLearning #Bioinformatics #ProteinModels #DrugDevelopment #Biopharmaceuticals #PLM #GNN #Biotech
Diffusion on language model encodings for protein sequence generation
1. DiMA is a novel continuous latent diffusion model that operates on protein language model (pLM) representations, enabling high-quality protein sequence generation across multiple encoder architectures.
2. Unlike traditional discrete diffusion and autoregressive models, DiMA generalizes across diverse encoders, including sequence-only models (ESM-2, ESMc), dual-decodable representations (CHEAP), and multimodal encoders (SaProt).
3. The model is computationally efficient, requiring only 35M parameters while matching or exceeding the performance of larger specialized models in various protein design tasks, including motif scaffolding and fold-conditioned sequence generation.
4. Through systematic architectural exploration, DiMA optimizes diffusion noise schedules and self-conditioning strategies, improving generation stability and accuracy compared to existing diffusion-based protein models.
5. Benchmarking results show that DiMA consistently produces diverse, high-quality, and novel protein sequences, outperforming alternative generative models like autoregressive, discrete diffusion, and flow-matching methods.
6. The model successfully applies conditional generation techniques, including protein family-specific design, motif scaffolding, and fold-specific sequence design, demonstrating its versatility in synthetic biology and protein engineering.
7. DiMA achieves strong distribution matching and high fidelity in generated sequences, validated against large-scale datasets such as SwissProt and AFDBv4-90, ensuring its applicability in real-world protein design scenarios.
📜Paper: https://t.co/PpPhWscuPc
#ProteinDesign #MachineLearning #DiffusionModels #ProteinEngineering #SyntheticBiology #ComputationalBiology #AI #Bioinformatics
Deep learning generates apo RNA conformations with cryptic ligand binding site
1. This study presents a novel application of deep learning (DL) to generate ligand-binding RNA conformations from apo RNA structures, enabling the discovery of cryptic binding sites without prior knowledge of the ligand-bound state.
2. By leveraging the Molearn deep learning framework, researchers generated HIV-1 TAR conformations with cryptic MV2003-binding cavities. These structures displayed binding scores comparable to experimentally resolved RNA-ligand complexes.
3. The Molearn framework maps RNA conformations onto a low-dimensional latent space, simplifying the representation of RNA conformational transitions and facilitating the exploration of energetically high, ligand-bindable conformations.
4. Cryptic binding sites, like the MV2003-binding cavity in HIV-1 TAR, are significant for drug discovery. These sites are absent in apo structures and only form under ligand influence, marking a shift in structure-based drug design (SBDD).
5. Using extensive molecular dynamics simulations and trained deep learning models, the study highlights the capability of DL to expand the conformational space of biomolecules beyond experimentally observed or simulated states.
6. Seven out of ten Molearn-trained models successfully generated TAR conformations that satisfied geometric and energetic criteria for ligand binding, demonstrating DL's potential to predict unprecedented RNA configurations.
7. The study integrates advanced RNA-specific metrics like RNA-BRiQ potential energy and geometrical analyses to validate the predicted conformations, bridging the gap between computational predictions and experimental relevance.
8. The approach provides a robust framework for RNA-targeted SBDD, emphasizing the importance of generating ligand-binding RNA conformations for evaluating drug candidates effectively.
@m_hama
💻Code: https://t.co/blBsukc7bJ
📜Paper: https://t.co/fBbcHsgs51
#RNA #DrugDiscovery #DeepLearning #SBDD #CrypticBindingSites
Free scientific illustrations for biologists! 😍
@NIH has released a library of 500+ free scientific illustrations to create figures, presentations, and illustrations!
all freely available in the public domain.
Retweet and spread the message!
https://t.co/p1bD1kxO7H
Zotero's new Note Editor can supercharge your note-taking and writing processes.
But many academics don't know about it.
Here's how to take notes with Zotero's Note Editor:
BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #NobelPrize in Chemistry with one half to David Baker “for computational protein design” and the other half jointly to Demis Hassabis and John M. Jumper “for protein structure prediction.”