Top Tweets for #molformer
Learning to Generalize: Deep Models and Robust Benchmarks for Drug–Target Affinity Prediction
1. This study introduces two novel sequence-based models, AffinityLM and AffinityNet, designed to predict drug–target binding affinity without relying on 3D structures. These models leverage FASTA and SMILES sequences to capture biochemical interactions, offering a scalable alternative to traditional methods.
2. The authors curated a new dataset of approximately 14,002 protein–ligand pairs, each mapped to a unique PDB ID and associated with experimentally determined binding affinities. This dataset addresses data leakage issues by ensuring no overlap of proteins and ligands across training, validation, and test sets.
3. AffinityNet, an attention-based architecture, integrates graph attention for local context and transformer blocks for long-range dependencies. It demonstrates superior performance, achieving a Pearson correlation coefficient of 0.67 and a mean absolute error of 5.8 kJ/mol, outperforming existing benchmarks.
4. AffinityLM utilizes embeddings from pre-trained biological–chemical language models (ESM-2 and MolFormer) to generate feature representations. While competitive, it shows limitations in generalizing to out-of-distribution datasets compared to AffinityNet.
5. The study highlights the importance of rigorous data splitting strategies, showing that traditional random splits can inflate model performance metrics by over 20% compared to more realistic cluster-based evaluations.
6. Independent tests on a molecular docking-based dataset and the CSAR-HiQ_36 dataset further validate AffinityNet's robust generalization capabilities, achieving correlations of 0.65 and 0.78 respectively, significantly outperforming other state-of-the-art models.
📜Paper: doi.org/10.26434/chemrxiv-20…
#DrugDiscovery #DeepLearning #Bioinformatics #ProteinLigand #BindingAffinity #AIinBiology

MolEncoder: Towards Optimal Masked Language Modeling for Molecules
1. A new study explores how to improve the prediction of molecular properties using machine learning models, particularly those based on transformer architectures inspired by natural language processing techniques. The research systematically investigates key factors like masking ratio, pretraining dataset size, and model size to optimize performance.
2. The study finds that higher masking ratios than commonly used (such as 30% instead of the typical 15%) significantly enhance the performance of molecular property prediction models. This is a crucial insight as it challenges the conventional wisdom borrowed from natural language processing and suggests that different design choices are needed for molecular data.
3. Contrary to the common assumption that larger models and pretraining datasets always lead to better performance, the research shows that increasing model size and dataset size beyond a certain point results in diminishing returns. In fact, a model with 15 million parameters pretrained on half of the ChEMBL dataset achieves the best performance, while larger models and datasets do not consistently offer benefits and can even degrade performance.
4. The authors developed MolEncoder, a BERT-based model that outperforms existing approaches on various drug discovery tasks while being more computationally efficient. This model achieves improved performance with significantly fewer computational resources, making it a more accessible and cost-effective solution for researchers with limited computational capabilities.
5. The study highlights the importance of carefully selecting pretraining settings for masked language modeling in molecular data. It provides practical guidelines for building more efficient molecular language models, which can help lower barriers for researchers and enable more efficient model development in the field of drug discovery.
6. The findings also show that pretraining loss is not a reliable predictor of downstream performance, emphasizing the need for direct evaluation on target tasks. This insight is valuable for guiding future research and development of molecular representation learning methods.
7. The study includes a comprehensive comparison of MolEncoder with other state-of-the-art models like ChemBERTa-2 and MoLFormer, demonstrating that MolEncoder achieves stronger and more consistent performance across multiple benchmark tasks despite using a smaller model size and pretraining dataset.
8. The authors make MolEncoder publicly available to support future work and encourage further research in this area. This open access to the model and its training datasets can facilitate collaboration and advancement in the field of molecular representation learning for drug discovery.
📜Paper: doi.org/10.26434/chemrxiv-20…
#MolEncoder #MaskedLanguageModeling #MolecularPropertyPrediction #DrugDiscovery #MachineLearning #ComputationalChemistry

Deep Learning vs Classical Methods in Potency & ADME Prediction: Insights from the Polaris Antiviral Challenge
1. A new study by computational biology experts sheds light on the performance of deep learning and classical machine learning methods in predicting compound properties. The key finding is that while classical methods are highly competitive for potency prediction, modern deep learning algorithms significantly outperform them in ADME (absorption, distribution, metabolism, excretion) prediction.
2. A critical takeaway from the analysis is the importance of data curation. The researchers demonstrated that by systematically identifying and masking "activity cliffs"—compounds with similar structures but large differences in activity—they could achieve a significant boost in predictive performance for both potency and ADME models.
3. The team also found that augmenting features with data from external public datasets led to improved predictive power, particularly for ADME endpoints. This highlights a valuable strategy for building more robust and accurate predictive models in drug discovery.
4. A surprising result was that learned molecular embeddings, generated by deep learning models like MolFormer and ChemBERTa, generally underperformed compared to classical "hand-crafted" descriptors. This suggests that while promising, these modern representations may still need further development to surpass established methods in certain applications.
5. Overall, the findings provide practical guidance for the field of computational drug discovery, emphasizing a hybrid approach: leveraging modern deep learning for ADME prediction, while recognizing the continued strength of classical methods and the crucial role of data curation and feature augmentation.
📜Paper: doi.org/10.26434/chemrxiv-20…
#ComputationalBiology #DrugDiscovery #MachineLearning #ADME #QSAR #DeepLearning #Cheminformatics

Transformer in Protein: A Survey
1.This comprehensive survey reviews over 100 studies applying Transformer models to protein informatics, providing the most extensive synthesis to date of how these architectures are transforming protein structure prediction, functional annotation, protein–protein interaction modeling, and drug discovery.
2.The paper introduces a domain-oriented classification system and thoroughly analyzes Transformer variants (e.g., AlphaFold, ProtTrans, ESM-Fold, ProteinBERT), elucidating their design innovations, training paradigms, and use across key biological tasks.
3.In protein structure prediction, models like AlphaFold, RoseTTAFold, and ESM-Fold are highlighted for revolutionizing 3D structure inference by capturing long-range residue dependencies and replacing costly experimental pipelines with Transformer-driven computation.
4.The review underscores how function prediction has been transformed by PLMs such as ProtBERT and ProtTrans, which generalize across remote homologs using self-supervised learning—enabling annotation for poorly characterized or novel proteins.
5.For protein–protein interaction (PPI) analysis, both sequence-based and structure-aware Transformer models are discussed, including hybrids like GACT-PPIS and graph-transformer models that outperform classical techniques in identifying binding sites and predicting interaction strength.
6.The survey also details how Transformer models are accelerating drug discovery by improving binding affinity prediction, virtual screening, and de novo molecular generation, especially with models like ChemBERTa, MolBERT, MolFormer, and graph-based generative Transformers.
7.A key contribution is the synthesis of Transformer derivatives tailored for proteins—ranging from SE(3)-equivariant attention to hybrid CNN-Transformer architectures—alongside detailed tabulations of model features, applications, and representative citations.
8.The paper organizes insights into a four-part structure: (1) Transformer fundamentals and protein-specific adaptations, (2) applications in protein structure/function/interaction/drug design, (3) key resources like benchmark datasets and open-source models, and (4) unresolved challenges and future directions.
9.Major benefits of Transformers in protein science include: modeling long-range dependencies, enabling multimodal integration (e.g., sequence + structure), supporting scalable pretraining on unlabeled data, and strong generalization across protein families.
10.Challenges remain in high computational cost, model interpretability, bias in protein databases, and generalization to rare proteins. The authors stress the need for hybrid models and improved explainability to broaden adoption in biomedical research.
11.Notably, the survey includes emerging applications such as cryo-EM-based structure inference (e.g., Cryo2Struct), MoRF region prediction, and use of Transformers with AutoML (e.g., MAGO) and explainable AI (e.g., SEGT-GO), signaling a shift toward more integrated and adaptive tools.
12.This paper serves as both an entry point for newcomers and a reference for experts, aiming to bridge AI innovation with protein science to advance bioinformatics, structural biology, and precision medicine.
📜Paper: arxiv.org/abs/2505.20098
#TransformersInBiology #ProteinStructure #AlphaFold #ProtTrans #PLMs #Bioinformatics #DrugDiscovery #PPIprediction #ProteinFunction #ProteinTransformerSurvey

DeepAden: an explainable machine learning for substrate specificity prediction in nonribosomal peptide synthetases
1.This paper introduces DeepAden, a two-stage explainable AI framework for predicting substrate specificity in NRPS A-domains, significantly outperforming prior methods, especially for non-proteinogenic substrates and low-data settings.
2.DeepAden uniquely integrates a graph attention network (ABP-GAT) for binding pocket prediction with cross-modal contrastive learning between protein pockets and substrate molecules, enabling accurate pairing even across divergent sequences.
3.The binding pocket predictor constructs a 2D protein graph from a 1D sequence using ESM2-derived embeddings and a ResNet-predicted contact map, identifying 27 key residues that form the binding pocket with high accuracy (F1 = 0.979).
4.The contrastive learning stage aligns binding pockets and substrates in a shared embedding space using ABP-ESM2 and MoLFormer, effectively capturing substrate preferences and enabling retrieval-based prediction for novel A-domains.
5.DeepAden introduces SHAP-based explainable AI to identify residue-level contributors to specificity. For example, it reveals Cys-specific ABPs utilize 18Gly and 20Ala, which enhance loop flexibility and favor distinct ligand conformations.
6.XAI-guided data augmentation allows targeted residue substitutions that preserve structural logic while enhancing model robustness, improving generalizability under few-shot and low-similarity scenarios.
7.Across three independent test sets (common, new, and few-shot substrates), DeepAden achieves the highest F1 scores (up to 0.822), improving by 12–80% over previous tools like NRPSpredictor2 and PKS/NRPS Analysis.
8.Unlike conventional tools relying on 10-AA or 34-AA SCC templates from a single species, DeepAden’s 27-AA ABP-based approach generalizes across evolutionary divergence and captures both proteinogenic and non-proteinogenic substrates.
9.In practical applications, DeepAden successfully mapped orphan biosynthetic gene clusters (BGCs) to experimentally validated compounds (nyuzenamide, octaminomycin), identifying multi-substrate A-domains missed by other predictors.
10.Interpretability is central to DeepAden: SHAP values highlight three key functional regions in ABPs—substrate entry, ATP-binding loop, and side chain contact zone—providing insights into evolution and enzyme engineering.
11.The model leverages pretrained PLMs and CLMs but freezes their weights, using only lightweight MLPs for projection, making DeepAden computationally efficient and easier to adapt across tasks.
12.By replacing alignment-dependent SCCs with interpretable, structure-informed sequence models, DeepAden paves the way for substrate-specific mining, rational NRPS design, and synthetic biology applications.
💻Code: github.com/Qinlab502/DeepAde…
📜Paper: biorxiv.org/content/10.1101/…
#NRPS #SubstrateSpecificity #ExplainableAI #DeepAden #ProteinDesign #BindingPocket #ContrastiveLearning #NaturalProductDiscovery #SyntheticBiology #ComputationalBiology

A Machine Learning Pipeline for Molecular Property Prediction using ChemXploreML
1.This work introduces ChemXploreML, a modular desktop application for predicting molecular properties using machine learning, offering flexibility, accessibility, and extensibility for chemists with varying levels of programming expertise.
2.The tool integrates multiple molecular embedding techniques (Mol2Vec and VICGAE) with powerful regression algorithms (GBR, XGBoost, CatBoost, LightGBM) to predict five key properties: melting point, boiling point, vapor pressure, critical temperature, and critical pressure.
3.Mol2Vec delivers slightly better predictive accuracy (up to R² = 0.93 for critical temperature), while VICGAE offers similar performance at a much lower dimensionality (32 vs. 300) and up to 10x faster computation—making it ideal for high-throughput screening.
4.ChemXploreML automates the entire pipeline: from chemical structure preprocessing and embedding generation to model training, hyperparameter optimization (via Optuna), and visualization of results through an intuitive desktop GUI.
5.Datasets were curated from the CRC Handbook of Chemistry and Physics, with SMILES representations canonicalized and cleaned using cleanlab to remove mislabeled or outlier data, ensuring high data quality for training robust models.
6.The system includes advanced visualization tools like UMAP for mapping high-dimensional chemical space, enabling users to explore structure–property trends and cluster molecules based on thermodynamic behavior.
7.Extensive benchmarking reveals that tree-based models paired with Mol2Vec embeddings excel in predicting boiling and critical temperatures, while VICGAE performs competitively—especially for challenging skewed properties like vapor pressure.
8.The modular framework supports future extensions, including ChemBERTa, MoLFormer, GNNs, and classification models, allowing researchers to tailor workflows to specific datasets or property types without code changes.
9.ChemXploreML's core contribution is bridging the gap between high-performance ML and practical chemistry workflows, offering an out-of-the-box platform for molecular design and property prediction in academic and industrial settings.
10.This study also highlights how embedding choice and property distribution shape model performance, with VICGAE emerging as a promising alternative to high-dimensional embeddings for fast, efficient learning on limited data.
💻Code: github.com/aravindhnivas/Che…
📜Paper: arxiv.org/abs/2505.08688v1
#Cheminformatics #MolecularPropertyPrediction #MachineLearning #Mol2Vec #Autoencoder #Regression #MLTools #ComputationalChemistry #UMAP #DataScience

ADDRESSING MODEL OVERCOMPLEXITY IN DRUG-DRUG INTERACTION PREDICTION WITH MOLECULAR FINGERPRINTS
1.This paper presents a strong case for simplicity in drug-drug interaction (DDI) prediction, showing that molecular fingerprints (MFPS) and pretrained GCN/MoLFormer embeddings can match or outperform much more complex graph or transformer-based models.
2.Using a shallow neural network fed with simple molecular embeddings, the authors achieve near state-of-the-art results across multiple DDI and drug-drug affinity (DDA) benchmarks, challenging the perceived need for deep architectures.
3.On the DrugBank benchmark, MFPS alone outperformed both knowledge graph models like KnowDDI and motif-aware models like MeTDDI, achieving an F1 score of 95.38 and Cohen’s κ of 94.50—higher than all other methods tested.
4.In the most challenging unseen-2-drugs DDI split, MFPS and pretrained GCN embeddings maintained comparable performance to SoTA models but with lower variance, showing greater prediction stability and generalizability.
5.In the DDA benchmark from FDA, MFPS achieved the best performance (RMSE = 0.838, PCC = 0.831), surpassing MeTDDI, SA-DDI, MolTrans, and transformer-based Molormer—all with less computational overhead.
6.The model is highly interpretable: gradient-based attribution on MFPS, GCN, and MoLFormer embeddings reveals chemically meaningful substructures (e.g., sulfur-nitrogen thiazoles) responsible for CYP3A4 inhibition in drugs like ritonavir and cobicistat.
7.Motif-level analysis confirmed that MFPS embeddings robustly capture known bioactive substructures (e.g., ccsc(C)n, cnc(cs)CN), while MoLFormer detects variants like Cc(n)s, offering complementary chemical insights.
8.Despite its simplicity, the model architecture generalizes well across all dataset conditions—including seen, one-drug unseen, and two-drug unseen splits—while maintaining high computational efficiency.
9.The authors emphasize that overly complex models often mask underlying data issues: limited chemical diversity, small size, and label noise remain key challenges in DDI/DDA benchmarks.
10.Their conclusion advocates for a progressive model design philosophy: start simple, evaluate generalizability, and only introduce complexity when justified by rigorous benchmarks and high-quality data.
11.This work serves as a blueprint for building interpretable, reproducible, and cost-effective ML models in biomedical applications—without sacrificing predictive power.
📜Paper: openreview.net/forum?id=RuEh…
#DDIPrediction #DrugDiscovery #MolecularFingerprints #MLforHealthcare #Bioinformatics #ModelSimplicity #ExplainableAI #ICLR2025 #GCN #MoLFormer #Pharmaceutics

LANTERN: TCR-Peptide Binding Prediction via Large Language Model Representations
1. LANTERN is a deep learning framework that leverages large language model embeddings to predict T-cell receptor (TCR) binding to peptide-MHC complexes, achieving high accuracy, especially on novel epitopes in zero-shot and few-shot settings.
2. Unlike traditional models, LANTERN integrates two powerful pretrained encoders: ESM for TCR sequences and MolFormer for SMILES-translated peptides, enhancing its ability to capture structural and chemical properties relevant to TCR recognition.
3. The use of SMILES encoding for peptides is a key innovation, transforming short sequences into chemically informative representations, significantly improving the model’s capacity to distinguish binding from non-binding pairs.
4. LANTERN outperforms both deep learning baselines and other LLM-based models like ChemBERTa, TEINet, and TITAN, achieving an AUC-ROC of 0.88 in realistic negative sampling settings, and showing a 23% performance gap over traditional models.
5. The model demonstrates strong generalization even with minimal data. In few-shot learning experiments, it rapidly learns binding patterns with fewer than 5 examples per peptide, highlighting its potential for applications with limited experimental data.
6. Embedding quality is quantitatively validated: pretraining significantly improves clustering metrics such as Silhouette Index, Calinski-Harabasz Index, Hopkins Score, and Cluster Tendency Score, showing tighter and more meaningful binding/non-binding separation.
7. The model uses hard data splits to ensure peptides in the test set are unseen during training, emphasizing robust generalization rather than memorization—a critical requirement for real-world immunotherapy development.
8. LANTERN’s performance is particularly strong in the more challenging random control setting, and its embedding clusters also reflect biological signal such as epitope frequency, indicating that it learns biologically meaningful features.
9. The framework’s architecture is modular and interpretable, relying on pretrained embeddings followed by a simple MLP classifier—making it adaptable to future improvements in molecular language models or additional data types.
10. LANTERN is a strong step forward for personalized immunotherapy, offering a scalable, data-efficient, and interpretable approach to modeling complex TCR-antigen interactions that are central to vaccine and T-cell therapy development.
💻Code: github.com/qi-c-lab/LANTERN
����Paper: arxiv.org/abs/2505.01433
#TCR #Immunotherapy #LLM #ProteinLanguageModel #Bioinformatics #MolFormer #ESM #TCell #DeepLearning #Immunology #LANTERN

Pre-trained molecular representations enable antimicrobial discovery @NatureComms
1. This study introduces MolE, a self-supervised framework that learns chemically meaningful molecular embeddings from unlabeled structures, enabling accurate antimicrobial activity prediction even with minimal labeled data.
2. MolE is pre-trained using only 100,000 unlabeled molecules and the Barlow-Twins redundancy reduction objective, creating rich task-agnostic representations that rival or outperform larger models like ChemBERTa and MolFormer.
3. By combining MolE with XGBoost, the authors developed a two-stage prediction pipeline (MolE-XGBoost) capable of estimating antimicrobial potential scores (AP scores) for any compound against 40 microbial strains, including human gut microbes.
4. MolE-XGBoost rediscovered Halicin and Abaucin—two structurally novel antibiotics—entirely from molecular structure alone, validating the model’s ability to generalize beyond known chemical scaffolds.
5. In experimental validation, three out of six predicted compounds (Elvitegravir, Opicapone, and Ebastine) exhibited growth-inhibitory activity against *Staphylococcus aureus*, with Elvitegravir showing strong inhibition at 8 μg/mL.
6. The AP scores correlate strongly (Spearman’s ρ = −0.5) with experimentally measured MIC values, confirming the framework’s ability to prioritize effective compounds and estimate potency.
7. Compared to ECFP4 fingerprints and descriptor-based models, MolE representations capture global and functional group-level features, allowing better discrimination of structural and functional similarity.
8. The approach correctly predicts broad-spectrum vs. narrow-spectrum activity and can rank compounds by predicted effectiveness against Gram-positive or Gram-negative strains using customized AP metrics (K, G, G+, G−).
9. MolE-XGBoost prioritized 158 non-antibiotic compounds with predicted broad-spectrum activity, 53 of which had prior experimental evidence—yielding a ~33% hit rate compared to the 1–3% in traditional screens.
10. The framework is computationally lightweight, requiring far less data and compute than traditional deep learning models, making it accessible for microbiologists with limited computational resources.
11. This work presents a scalable, interpretable, and experimentally validated method for antimicrobial discovery that complements high-throughput screening and opens new directions for drug repurposing and resistance research.
💻Code: github.com/rolayoalarcon/Mol…
📜Paper: nature.com/articles/s41467-0…
#AntimicrobialResistance #DrugDiscovery #MolecularEmbeddings #SelfSupervisedLearning #Bioinformatics #MolE #Antibiotics #MachineLearning #GraphNeuralNetworks #Chemoinformatics #Microbiome

ADDRESSING MODEL OVERCOMPLEXITY IN DRUG-DRUG INTERACTION PREDICTION WITH MOLECULAR FINGERPRINTS
1. This study challenges the trend of increasingly complex DDI prediction models by showing that simple, interpretable molecular representations—like Morgan fingerprints (MFPS)—can match or outperform state-of-the-art graph- and knowledge-based methods.
2. On multiple DrugBank benchmarks, MFPS achieve top or statistically comparable performance to advanced models such as MeTDDI and KnowDDI, while requiring significantly less computational cost and offering better interpretability.
3. Even under the strictest split—where both drugs in a test pair are unseen during training—MFPS maintain competitive accuracy and lower performance variance, suggesting better generalizability in real-world scenarios.
4. MFPS consistently outperform complex alternatives like MolFormer and pretrained GCNs in drug-drug affinity prediction on the FDA dataset, achieving the highest Pearson correlation (0.831) and lowest RMSE (0.838).
5. Using gradient-based attribution, the study highlights how MFPS can pinpoint chemically meaningful motifs (e.g., sulfur-nitrogen thiazole cores in ritonavir and cobicistat) relevant to CYP3A4 inhibition, validating both biological plausibility and model transparency.
6. Compared to knowledge-driven models like SumGNN and KnowDDI, MFPS embeddings achieve higher F1, accuracy, and Cohen's kappa scores—demonstrating that simple representations can still excel even without access to curated relational graphs.
7. The authors built a streamlined model architecture with shallow neural networks and dropout regularization, keeping training stable and resource-efficient across all tasks (classification and regression).
8. While recognizing that complex models have their place, the authors argue for progressive complexity scaling—starting from simple baselines like theirs—especially in the presence of noisy, small, or low-diversity datasets.
9. This work underscores the importance of dataset quality. Many current DDI datasets are chemically redundant or label-inconsistent, making high-capacity models appear better than they truly generalize.
10. Overall, the paper promotes MFPS as a robust, interpretable baseline for DDI/DDA research and calls for more careful evaluation standards before defaulting to deep, opaque architectures.
💻Code: github.com/nostrumbiodiscove…
📜Paper: arxiv.org/abs/2503.23550
#DrugDiscovery #Cheminformatics #DDI #AI4Science #InterpretableML #MolFingerprint #Bioinformatics #Pharmacology #DeepLearning

How to generalize machine learning models to both canonical and non-canonical peptides
1. This paper presents a systematic evaluation of machine learning models for predicting peptide functions across both canonical and non-canonical peptides, addressing the challenge of generalization between these two classes.
2. It highlights the limitations of existing methods, which struggle to generalize from canonical peptide data to non-canonical peptides due to differences in structural diversity and limited data availability.
3. The authors introduce a novel approach that utilizes similarity-based dataset partitioning with MAPc and ECFP fingerprints for non-canonical peptides and MMSeqs2 for canonical peptides, optimizing data splits to ensure robust performance.
4. They find that fingerprint-based similarity measures, especially MAPc and ECFP, outperform traditional sequence alignment methods for partitioning canonical datasets, suggesting a shift from conventional practices.
5. To address the problem of representation, the study benchmarks several molecular representation methods, including Chemical Language Models (CLMs) like ChemBERTa-2 and Molformer-XL, protein language models like ESM2 and Prot-T5-XL, and molecular fingerprints like ECFP and PepFuNN.
6. Among all tested representations, general-purpose models like ChemBERTa-2 and Molformer-XL performed better than peptide-specific models such as PeptideCLM and Pepland, with ChemBERTa-2 showing the best generalization across both canonical and non-canonical datasets.
7. The authors demonstrate that training models solely on canonical or non-canonical data results in poor generalization to the other class. However, training on mixed datasets enables successful generalization with minimal performance loss.
8. Jointly trained models effectively generalize to both canonical and non-canonical peptides, even when the dataset is heavily imbalanced, suggesting that representation techniques can effectively bridge the gap between these two peptide classes.
9. The study concludes that ChemBERTa-2 is the most suitable representation technique for joint modeling and recommends further exploration of hybrid approaches combining different representation techniques for better performance.
10. The authors release benchmark datasets and partitioning schemes to aid future research on peptide representation learning.
💻Code: github.com/IBM/AutoPeptideML…
📜Paper: doi.org/10.26434/chemrxiv-20…
#PeptideModeling #MachineLearning #Bioinformatics #CanonicalPeptides #NonCanonicalPeptides #ChemBERTa #Generalization #MolecularRepresentations #PeptideRepresentationLearning

Pre-trained Molecular Language Models with Random Functional Group Masking
• This paper introduces MLM-FG, a SMILES-based molecular language model that employs random masking of functional groups to enhance molecular property prediction. Unlike traditional models that mask random subsequences, MLM-FG focuses on key chemical substructures, leading to a deeper understanding of molecular features.
• MLM-FG’s functional group-aware masking strategy allows it to infer structural information implicitly, outperforming both SMILES- and graph-based models, including those with explicit 3D information, on 9 out of 11 benchmark tasks.
• Utilizing a transformer-based architecture, MLM-FG captures molecular representations by encoding functional groups, making it exceptionally effective in tasks like drug discovery, where structural nuances are critical.
• Experiments on MoleculeNet benchmarks, covering classification and regression tasks, show MLM-FG achieves superior predictive performance over leading models such as MoLFormer and GEM, demonstrating significant gains in prediction accuracy.
• The model’s capability to learn structural relationships directly from SMILES strings, without 3D data, highlights its potential to improve molecular analysis and reduce data annotation costs, making it suitable for large-scale applications.
💻Code: anonymous.4open.science/r/ML…
📜Paper: arxiv.org/abs/2411.01401
#MolecularLanguageModel #FunctionalGroup #SMILES #DrugDiscovery #Bioinformatics

SELF-BART: A Transformer-based Molecular Representation Model using SELFIES @IBMResearch
1. SELF-BART introduces an encoder-decoder architecture that leverages SELFIES (SELF-referencing Embedded Strings) to ensure the generation of syntactically valid molecules, outperforming SMILES-based approaches.
2. The model addresses a key limitation in existing molecular models by combining representation learning with auto-regressive generation, making it suitable for both molecular property prediction and molecule generation tasks.
3. SELF-BART’s use of a denoising objective during training improves its ability to learn robust molecular representations, leading to superior performance across nine MoleculeNet benchmark datasets.
4. Compared to graph-based and SMILES-based models like ChemBERTa and MolFormer, SELF-BART achieves state-of-the-art results in classification tasks (e.g., 96.9 ROC-AUC on ClinTox) and regression tasks (e.g., 1.397 RMSE on FreeSolv).
5. The model demonstrates excellent generative capabilities, achieving 99.8% validity, 100% novelty, and high internal diversity in molecule generation tasks, showcasing its potential for drug discovery applications.
6. SELF-BART’s encoder-decoder framework provides the flexibility to generate new molecules from learned representations, allowing researchers to explore novel chemical spaces effectively.
7. Preliminary results indicate SELF-BART’s potential to generate diverse, valid, and novel molecular structures, setting the stage for future studies on conditioned molecular generation.
8. This study underscores the advantages of SELFIES for molecular modeling and highlights SELF-BART’s applicability in both virtual screening and molecular optimization tasks.
📜Paper: arxiv.org/abs/2410.12348

Most Popular Users

Elon Musk 
@elonmusk
240.3M followers

Barack Obama 
@barackobama
118.6M followers

Cristiano Ronaldo 
@cristiano
114.4M followers

Donald J. Trump 
@realdonaldtrump
111.8M followers

Narendra Modi 
@narendramodi
106.9M followers

Rihanna 
@rihanna
98.6M followers

NASA 
@nasa
92.2M followers

Justin Bieber 
@justinbieber
91.7M followers

KATY PERRY 
@katyperry
89.9M followers

Taylor Swift 
@taylorswift13
83.9M followers

Lady Gaga 
@ladygaga
75.4M followers

Virat Kohli 
@imvkohli
73.3M followers

Kim Kardashian 
@kimkardashian
70.8M followers

YouTube 
@youtube
68.6M followers

Neymar Jr 
@neymarjr
66.3M followers

Bill Gates 
@billgates
65.2M followers

Selena Gomez 
@selenagomez
63M followers

The Ellen Show
@theellenshow
62.1M followers

CNN 
@cnn
61.7M followers

X 
@x
60.6M followers
