Top Tweets for #ChemBERTa
DMAPLM: A multimodal pretrained framework for computational drug repositioning.
1 DMAPLM reframes drug–disease association prediction as cross-modal representation alignment: a lightweight dual-encoder maps drug SMILES (ChemBERTa-2) and disease multi-field text (BioBERT) into a shared space, improving generalization when curated associations are sparse.
2 The most distinctive design choice is explicit contrastive learning (InfoNCE) between drug and disease embeddings, encouraging true drug–disease pairs to be close while pushing unrelated pairs apart; this yields markedly better separability (e.g., higher positive-pair cosine similarity and lower negative-pair similarity) and supports similarity-based interpretability.
3 For drugs, DMAPLM avoids graph construction and instead uses attention-weighted pooling over ChemBERTa-2 token embeddings, letting the model emphasize informative SMILES tokens/substructures; attention visualization highlights pharmacologically relevant motifs (e.g., rapamycin’s macrolide core/epoxide consistent with FKBP12–mTOR binding chemistry).
4 For diseases, it encodes three complementary text fields (name, synonyms, definition) separately with BioBERT and fuses them via weighted aggregation, then projects to a compact embedding; clustering analyses show these PLM disease embeddings align better with MeSH taxonomy than TF-IDF or random baselines.
5 The final prediction step is intentionally simple and robust: a Random Forest classifier operates on concatenated multimodal features (raw + projected embeddings + interaction features), and a Youden-index threshold converts probabilities to binary calls for practical screening.
6 The authors also contribute an updated benchmark curated from DrugMAP 2.0 (Approved associations only): 2,622 drugs, 1,455 diseases, 5,993 associations, with very low density (0.16%), reflecting realistic incompleteness and making generalization a central challenge.
7 On 5-fold CV, DMAPLM reports AUROC 0.8919 and AUPR 0.9116, outperforming six baselines (including graph-based and heterogeneous transformer methods) by up to ~9% on AUROC/AUPR, suggesting the PLM+alignment strategy can compete with heavier graph pipelines.
8 Cold-start evaluation follows standard C1/C2/C3 protocols: new drugs (C1) AUROC/AUPR 0.8150/0.8338, new diseases (C2) 0.7805/0.8098, and double cold-start (C3) 0.7456/0.7355—highlighting the benefit of pretrained encoders when entities lack training-time links.
9 Biological plausibility is supported via case studies and docking: top cold-start predictions (e.g., rapamycin–prostate cancer) are backed by PubMed evidence, and docking against the FKBP12-rapamycin-FRB mTOR complex (PDB 4FAP) yields a best Vina affinity of -9.921 kcal/mol with consistent poses, reinforcing mechanistic consistency rather than purely statistical ranking.
💻Code: github.com/LiNiuNiuYa/DMAPLM
📜Paper: doi.org/10.1371/journal.pcbi…
#DrugRepositioning #DrugRepurposing #ComputationalBiology #BiomedicalNLP #Chemoinformatics #ContrastiveLearning #BioBERT #ChemBERTa #MultimodalAI #ColdStart

ReACT-Drug: Reaction-Template Guided Reinforcement Learning for de novo Drug Design
1. ReACT-Drug presents a novel reinforcement learning framework for de novo drug design that is target-agnostic, eliminating the need for extensive target-specific training. This approach leverages ESM-2 protein embeddings to identify similar proteins and their ligands, initializing a fragment-based search space for generating new drug candidates.
2. The framework uses a Proximal Policy Optimization (PPO) agent to guide molecular transformations based on a dynamic action space of reaction templates. This ensures that the generated molecules are chemically valid and novel, achieving 100% chemical validity and novelty according to MOSES benchmarking.
3. ReACT-Drug integrates structural biology, deep representation learning, and chemical synthesis rules to automate drug design. It demonstrates superior or competitive performance compared to existing models across six diverse protein targets, including serotonin receptors, kinases, and opioid receptors.
4. The system achieves binding affinities ranging from -9.13 to -10.4 kcal/mol across different targets, with some candidates approaching or exceeding database baselines. This highlights its potential for rapid exploration of novel chemical space without target-specific fine-tuning.
5. The dataset and code are available for further exploration and validation, making this a valuable resource for researchers in the field of computational drug design.
📜Paper: arxiv.org/abs/2512.20958v1
💻Code: github.com/YadunandanRaman/R…
#drugDesign #reinforcementLearning #ESM2 #ChemBERTa #molecularGeneration #deNovoDrugs

DlRNA-BERTa: A Transformer Approach for RNA-Drug Binding Affinity Prediction
1. DlRNA-BERTa is a novel RoBERTa-based framework that combines RNABERTa, pretrained on 9.76 million RNA sequences, with ChemBERTa-v2 to predict small molecule–RNA interactions. It includes six class-specific models for different RNA types and a general model for unknown RNA classes, outperforming existing RNA–drug interaction prediction methods with Pearson correlation coefficients up to 0.98 for miRNAs.
2. The study leverages a cross-attention mechanism to capture contextual interactions between RNA and drug tokens, enabling end-to-end, structure-free prediction of RNA–drug binding affinities. This approach allows the model to achieve robust performance across various RNA classes, even those with limited training data, such as repeats and riboswitches.
3. Application of DlRNA-BERTa to 3,492 approved drugs from the ChEMBL database identified 2,859 compounds with predicted affinities (pKd ≥6) across 294 RNA targets. Notably, bleomycin was highlighted with literature evidence supporting its RNA-binding activity, demonstrating the model's biological relevance and predictive reliability.
4. The model's architecture and training process incorporate several innovations, including the use of μParametrization for scalable hyperparameter transferability and the optimization of hyperparameters using Optuna with the Tree-structured Parzen Estimator (TPE) sampler. These enhancements contribute to the model's efficiency and performance.
5. A publicly accessible web application is available at huggingface.co/spaces/IlPako…, providing user-friendly access to the general model. The source code and datasets are openly available at github.com/IlPakoZ/rnaberta-…, in accordance with FAIR principles, allowing researchers to extend the framework and advance RNA-targeted drug discovery.
📜Paper: biorxiv.org/content/10.1101/…
#RNAtherapeutics #DrugDiscovery #TransformerModels #Bioinformatics #AIinMedicine

MolEncoder: Towards Optimal Masked Language Modeling for Molecules
1. A new study explores how to improve the prediction of molecular properties using machine learning models, particularly those based on transformer architectures inspired by natural language processing techniques. The research systematically investigates key factors like masking ratio, pretraining dataset size, and model size to optimize performance.
2. The study finds that higher masking ratios than commonly used (such as 30% instead of the typical 15%) significantly enhance the performance of molecular property prediction models. This is a crucial insight as it challenges the conventional wisdom borrowed from natural language processing and suggests that different design choices are needed for molecular data.
3. Contrary to the common assumption that larger models and pretraining datasets always lead to better performance, the research shows that increasing model size and dataset size beyond a certain point results in diminishing returns. In fact, a model with 15 million parameters pretrained on half of the ChEMBL dataset achieves the best performance, while larger models and datasets do not consistently offer benefits and can even degrade performance.
4. The authors developed MolEncoder, a BERT-based model that outperforms existing approaches on various drug discovery tasks while being more computationally efficient. This model achieves improved performance with significantly fewer computational resources, making it a more accessible and cost-effective solution for researchers with limited computational capabilities.
5. The study highlights the importance of carefully selecting pretraining settings for masked language modeling in molecular data. It provides practical guidelines for building more efficient molecular language models, which can help lower barriers for researchers and enable more efficient model development in the field of drug discovery.
6. The findings also show that pretraining loss is not a reliable predictor of downstream performance, emphasizing the need for direct evaluation on target tasks. This insight is valuable for guiding future research and development of molecular representation learning methods.
7. The study includes a comprehensive comparison of MolEncoder with other state-of-the-art models like ChemBERTa-2 and MoLFormer, demonstrating that MolEncoder achieves stronger and more consistent performance across multiple benchmark tasks despite using a smaller model size and pretraining dataset.
8. The authors make MolEncoder publicly available to support future work and encourage further research in this area. This open access to the model and its training datasets can facilitate collaboration and advancement in the field of molecular representation learning for drug discovery.
📜Paper: doi.org/10.26434/chemrxiv-20…
#MolEncoder #MaskedLanguageModeling #MolecularPropertyPrediction #DrugDiscovery #MachineLearning #ComputationalChemistry

Deep Learning vs Classical Methods in Potency & ADME Prediction: Insights from the Polaris Antiviral Challenge
1. A new study by computational biology experts sheds light on the performance of deep learning and classical machine learning methods in predicting compound properties. The key finding is that while classical methods are highly competitive for potency prediction, modern deep learning algorithms significantly outperform them in ADME (absorption, distribution, metabolism, excretion) prediction.
2. A critical takeaway from the analysis is the importance of data curation. The researchers demonstrated that by systematically identifying and masking "activity cliffs"—compounds with similar structures but large differences in activity—they could achieve a significant boost in predictive performance for both potency and ADME models.
3. The team also found that augmenting features with data from external public datasets led to improved predictive power, particularly for ADME endpoints. This highlights a valuable strategy for building more robust and accurate predictive models in drug discovery.
4. A surprising result was that learned molecular embeddings, generated by deep learning models like MolFormer and ChemBERTa, generally underperformed compared to classical "hand-crafted" descriptors. This suggests that while promising, these modern representations may still need further development to surpass established methods in certain applications.
5. Overall, the findings provide practical guidance for the field of computational drug discovery, emphasizing a hybrid approach: leveraging modern deep learning for ADME prediction, while recognizing the continued strength of classical methods and the crucial role of data curation and feature augmentation.
📜Paper: doi.org/10.26434/chemrxiv-20…
#ComputationalBiology #DrugDiscovery #MachineLearning #ADME #QSAR #DeepLearning #Cheminformatics

HTS-Oracle: A Retrainable AI Platform for High-Confidence Hit Identification Across Difficult-to-Drug Targets
1. The introduction of HTS-Oracle marks a significant advancement in high-throughput screening (HTS) for drug discovery. This AI platform leverages deep learning to integrate molecular embeddings from ChemBERTa with traditional cheminformatics features, offering a powerful ensemble framework for predicting high-confidence hits across challenging targets.
2. One of the key innovations of HTS-Oracle is its ability to reprioritize and streamline the screening process. By enriching true positives and filtering out non-binders upfront, it significantly reduces the experimental burden and improves the efficiency of hit identification, particularly for difficult-to-drug targets like the immune co-stimulatory receptor CD28.
3. The platform demonstrated an impressive 8.4% hit rate in prospective screening, representing an eightfold improvement over conventional methods such as surface plasmon resonance (SPR) and affinity selection mass spectrometry (ASMS). This highlights HTS-Oracle’s potential to transform early-stage drug discovery by prioritizing high-confidence hits from large compound libraries.
4. HTS-Oracle’s architecture is designed for retrainability and adaptability. It can be retrained for new targets, making it a scalable and practical tool for iterative library refinement and target-specific optimization. This flexibility is crucial for addressing the diverse challenges in drug discovery across different therapeutic areas.
5. The study also validated the platform’s predictions experimentally, identifying two compounds that disrupted the CD28–B7.1 interaction with micromolar potency. These findings underscore the platform’s ability to identify novel small molecule inhibitors, even for targets historically considered intractable.
6. HTS-Oracle is released as an open-source platform, complete with training code and a pre-trained model. This accessibility encourages broader adoption and further development within the scientific community, providing a strong foundation for integrating AI into prospective screening workflows.
📜Paper: biorxiv.org/content/10.1101/…
💻Code: github.com/HTS-Oracle/CD28-p…
#HTSOracle #AIDrugDiscovery #HighThroughputScreening #MachineLearning #DrugDiscovery #OpenSource #AIPlatform

MorganBERT: Chemical substructure sequences meet language models for property prediction
1. MorganBERT is a novel approach that leverages chemical substructure sequences to improve the prediction of molecular properties using language models. This method stands out by utilizing the extended-connectivity fingerprints (ECFP) algorithm to generate unique identifiers for chemical substructures, which are then used as input for the RoBERTa architecture.
2. The core innovation of MorganBERT lies in its ability to capture the local chemical environment around heavy atoms in molecules. By treating these atom identifiers as “words” and sequences of identifiers as “sentences,” the model can effectively learn from over 52 million compounds, significantly enhancing its predictive capabilities.
3. MorganBERT outperforms existing methods, including traditional fingerprints, graph neural networks (GNNs), and other chemical language models (CLMs), across multiple tasks relevant to drug development and toxicology. This includes predicting ADME properties, CYP450 interactions, and acute toxicity endpoints.
4. A key advantage of MorganBERT is its ability to organize chemicals along a continuous gradient in relation to their activity values. This approximate isometry between chemical space and activity space allows for better interpretability of predictions, making it easier to understand how structural changes impact biological activities.
5. The study demonstrates MorganBERT’s effectiveness through benchmarking against diverse methods. For instance, in predicting aqueous solubility, MorganBERT achieved a Pearson correlation coefficient (PCC) of up to 0.929, surpassing previous models like ChemBERTa and MolFormer.
6. MorganBERT’s unique approach also enables the generation of explanations for individual molecule predictions through counterfactuals. By exploring the local chemical space surrounding a focal molecule, the model can identify structurally similar molecules with different predicted outcomes, providing insights into the factors influencing activity.
7. The authors highlight that MorganBERT’s pretraining on a large corpus of chemical structures allows it to transfer learned structural information to specific tasks, mitigating the limitations of narrow applicability domains (ADs) often seen in traditional QSAR models.
📜Paper: doi.org/10.26434/chemrxiv-20…
#MorganBERT #ChemicalLanguageModel #QSAR #DrugDiscovery #Toxicology #AIinChemistry

ChemBERTa-3: An Open Source Training Framework for Chemical Foundation Models
A new open-source training framework, ChemBERTa-3, has been introduced to address the challenge of building scalable and robust models for molecular property prediction. This framework provides a unified and extensible platform for training and benchmarking large-scale chemical foundation models.
A key innovation of ChemBERTa-3 is its rigorous, standardized benchmarking approach. It tackles inconsistencies found in prior studies by standardizing scaffold splitting and evaluation protocols, ensuring more reliable comparisons across diverse model architectures. This is critical for advancing reproducible science in computational chemistry.
The framework is fully integrated into the DeepChem library, leveraging its extensive collection of models and benchmarking infrastructure. It introduces new components like ModularTorchModel for simplified model development and a HuggingFace Deepchem Wrapper for seamless integration of transformer models.
ChemBERTa-3 demonstrates impressive scalability, designed to train and fine-tune models on billions of molecules. It achieves this by utilizing AWS-based Ray deployments and on-premise high-performance computing clusters, overcoming significant computational hurdles in the field.
Experiments conducted within the framework provide valuable insights, indicating that transformer-based models, when pre-trained on large datasets, can achieve performance comparable to graph-based approaches while offering superior scalability and generalization capabilities.
In the spirit of open science, all ChemBERTa-3 models and the underlying code have been open-sourced. This initiative aims to foster further research and development in the science of chemical foundation models, promoting transparency and collaboration within the community.
💻Code: github.com/deepforestsci/che…
📜Paper: chemrxiv.org/engage/chemrxiv…
#ComputationalChemistry #MachineLearning #DrugDiscovery #ChemicalModels #OpenScience #DeepChem #AIinChemistry

BitterTranslate: A Natural Language Processing and Machine Learning-based Framework for Mapping Bitter Taste Receptor Agonism
1.BitterTranslate is a machine learning framework designed to predict interactions between small molecules and human bitter taste receptors (TAS2Rs), using only a SMILES string and a receptor amino acid sequence as input.
2.It leverages transformer-based language models (ChemBERTa for molecules and a custom-trained GPCRBERT for receptors) to extract high-dimensional embeddings, bypassing the need for manual feature engineering or proprietary software.
3.The final model uses an XGBoost classifier pre-trained on over 40,000 GPCR–ligand associations and fine-tuned on 5,800 TAS2R–ligand interactions. This transfer learning approach significantly improves recall.
4.BitterTranslate achieves 80% precision and 65% recall across 21 TAS2Rs, outperforming the previous state-of-the-art method, BitterMatch, especially in recall and F1 score.
5.For broadly tuned receptors like TAS2R14, the model performs exceptionally well: 83% precision, 88% recall, and 91% average precision on the PR curve.
6.The tool demonstrates strong generalizability, achieving 88% recall and 79% precision on a diverse bioassay dataset involving non-TAS2R GPCRs.
7.Feature vectors include over 4,000 dimensions, incorporating receptor loop-specific embeddings (especially ECL2) and comprehensive molecular descriptors from RDKit.
8.The model shows robust precision even for poorly annotated receptors, though recall declines when receptor data is sparse—highlighting data quantity as a limiting factor.
9.The framework is available as a standalone software tool that accepts a CSV of SMILES strings and outputs predicted TAS2R associations, supporting high-throughput screening.
10.Authors stress the complementarity of BitterTranslate with traditional physics-based modeling and experimental validation, not as a replacement but as a high-throughput filtering tool.
11.They highlight potential future directions including regression-based prediction of binding affinity (pending high-quality EC50 data), TAS2R antagonist modeling, and broader GPCR family applications.
12.BitterTranslate is a step toward efficient computational pre-screening in drug discovery and taste research, where experimental throughput is a major bottleneck.
📜Paper: doi.org/10.26434/chemrxiv-20…
#GPCR #DrugDiscovery #MachineLearning #NLP #Transformers #BitterTaste #ChemBERTa #ProtBERT #ComputationalBiology #Bioinformatics

Accelerating ligand discovery by combining Bayesian optimization with MMGBSA-based binding affinity calculations
1.This study introduces a scalable active learning framework that integrates Bayesian optimization with MMGBSA-based binding affinity predictions, offering MD-level accuracy in ligand ranking at significantly reduced computational cost.
2.Unlike traditional docking scores, MMGBSA demonstrates a much higher correlation with experimental binding affinities (Pearson R = 0.61 vs 0.18 for Vina), making it a more reliable metric for virtual screening.
3.The pipeline replaces human-guided candidate selection with a data-driven strategy using pretrained molecular embeddings (MolFormer, ChemBERTa-2) and lightweight surrogate models (Bayesian linear regression, Random Forests).
4.A one-at-a-time acquisition strategy in active learning outperforms traditional batch approaches, retrieving up to 80% of the top-1% binders (by MMGBSA score) after querying just 6% of a 60,000-compound dataset.
5.Surprisingly, the simple linear regression model consistently outperforms more complex Random Forests, attributed to better-calibrated uncertainty estimates essential for effective Bayesian optimization.
6.Pretrained embeddings from MolFormer outperform ChemBERTa-2 when learning MMGBSA scores, likely due to the richer structural representations learned through masked language modeling.
7.Learning MMGBSA scores is significantly harder than learning docking scores, likely due to their complex dependence on 3D structural and dynamic properties, which SMILES-based embeddings alone may not fully capture.
8.Even though docking-based models can achieve higher retrieval rates (up to 97%), MMGBSA-based selection retrieves chemically distinct compounds that are more likely to exhibit true high binding affinity.
9.The framework successfully scales to large chemical spaces, as demonstrated with a 60k-compound subset targeting the MCL1 protein, a challenging cancer target with known binders.
10.Performance is sensitive to the initial molecule used to seed the active learning loop; however, this sensitivity can be mitigated by smarter initialization or diverse starting points.
11.Unlike prior methods, the authors show that retraining only the regression head is sufficient, allowing fast and frequent model updates—critical for practical deployment in high-performance computing environments.
12.Future directions include incorporating richer 3D or pose-aware features into molecular embeddings and extending the pipeline to more rigorous free energy methods or experimental feedback.
💻Code: github.com/maxrdu/bayesian-o…
📜Paper: biorxiv.org/content/10.1101/…
#DrugDiscovery #ActiveLearning #MolecularDocking #MMGBSA #BayesianOptimization #Cheminformatics #ProteinLigandBinding

ChemBERTaDDI: Transformer Driven Molecular Structures and Clinical Data for predicting drug-drug interactions
1.ChemBERTaDDI proposes a unified framework that integrates transformer-derived molecular embeddings with clinical side effect data to predict adverse drug-drug interactions (DDIs). It achieves strong performance with minimal architectural complexity, outperforming existing state-of-the-art methods.
2.At its core, the model uses ChemBERTa-77M-MLM, a transformer trained on 77 million SMILES strings, to encode rich, contextual chemical structure information into 384-dimensional embeddings.
3.Each drug is also represented by a 595-dimensional PCA-compressed vector of mono side effects from SIDER and OFFSIDES databases, capturing clinical phenotype data. These two modalities are concatenated into a 979-dimensional vector.
4.For a drug pair, their vectors are summed to produce a joint representation and passed into a three-layer neural network for binary classification, predicting the likelihood of a specific adverse interaction among 964 polypharmacy side effects.
5.The approach avoids reliance on handcrafted features or noisy graph structures and instead leverages pretrained contextual embeddings and clinical statistics, ensuring robustness and scalability.
6.Extensive benchmarking shows ChemBERTaDDI outperforms Decagon, DeepDDI, MDF-SA-DDI, DPSP, and NNPS in F1 score, AUPR, and AUROC. Notably, it achieved an F1 score of 0.938 and AUPR of 0.954, generalizing well to unseen drug pairs.
7.In high-risk side effect categories such as sarcoma and malignant hypertension, ChemBERTaDDI demonstrates superior predictive precision compared to Decagon, with AUPRs of 0.978 and 1.0 respectively.
8.Ablation studies show that combining chemical and clinical features significantly boosts performance, while adding protein data offers negligible improvement, emphasizing the power of just these two modalities.
9.Even when using only SMILES-based embeddings, the model performs competitively (F1 ~0.891), indicating that ChemBERTa-derived features alone capture substantial molecular semantics.
10.The model uses a simple yet effective prediction architecture: three dense layers (250–300–200 neurons) with batch norm, ReLU, and dropout, followed by a sigmoid output for binary interaction prediction.
11.Training leveraged the TWOSIDES dataset for DDI labels and PubChem for SMILES, covering 645 drugs and 63,473 pairs, filtered down to 964 most frequent side effects for training and evaluation.
12.The study provides a well-documented and reproducible pipeline with publicly available code, making it practical for adaptation in drug safety research and pharmaceutical pipelines.
💻Code: github.com/anastasiyagromova…
📜Paper: biorxiv.org/content/10.1101/…
#DrugSafety #DeepLearning #Transformers #Bioinformatics #DrugDiscovery #ChemBERTa #DDI #MachineLearning

Transformer in Protein: A Survey
1.This comprehensive survey reviews over 100 studies applying Transformer models to protein informatics, providing the most extensive synthesis to date of how these architectures are transforming protein structure prediction, functional annotation, protein–protein interaction modeling, and drug discovery.
2.The paper introduces a domain-oriented classification system and thoroughly analyzes Transformer variants (e.g., AlphaFold, ProtTrans, ESM-Fold, ProteinBERT), elucidating their design innovations, training paradigms, and use across key biological tasks.
3.In protein structure prediction, models like AlphaFold, RoseTTAFold, and ESM-Fold are highlighted for revolutionizing 3D structure inference by capturing long-range residue dependencies and replacing costly experimental pipelines with Transformer-driven computation.
4.The review underscores how function prediction has been transformed by PLMs such as ProtBERT and ProtTrans, which generalize across remote homologs using self-supervised learning—enabling annotation for poorly characterized or novel proteins.
5.For protein–protein interaction (PPI) analysis, both sequence-based and structure-aware Transformer models are discussed, including hybrids like GACT-PPIS and graph-transformer models that outperform classical techniques in identifying binding sites and predicting interaction strength.
6.The survey also details how Transformer models are accelerating drug discovery by improving binding affinity prediction, virtual screening, and de novo molecular generation, especially with models like ChemBERTa, MolBERT, MolFormer, and graph-based generative Transformers.
7.A key contribution is the synthesis of Transformer derivatives tailored for proteins—ranging from SE(3)-equivariant attention to hybrid CNN-Transformer architectures—alongside detailed tabulations of model features, applications, and representative citations.
8.The paper organizes insights into a four-part structure: (1) Transformer fundamentals and protein-specific adaptations, (2) applications in protein structure/function/interaction/drug design, (3) key resources like benchmark datasets and open-source models, and (4) unresolved challenges and future directions.
9.Major benefits of Transformers in protein science include: modeling long-range dependencies, enabling multimodal integration (e.g., sequence + structure), supporting scalable pretraining on unlabeled data, and strong generalization across protein families.
10.Challenges remain in high computational cost, model interpretability, bias in protein databases, and generalization to rare proteins. The authors stress the need for hybrid models and improved explainability to broaden adoption in biomedical research.
11.Notably, the survey includes emerging applications such as cryo-EM-based structure inference (e.g., Cryo2Struct), MoRF region prediction, and use of Transformers with AutoML (e.g., MAGO) and explainable AI (e.g., SEGT-GO), signaling a shift toward more integrated and adaptive tools.
12.This paper serves as both an entry point for newcomers and a reference for experts, aiming to bridge AI innovation with protein science to advance bioinformatics, structural biology, and precision medicine.
📜Paper: arxiv.org/abs/2505.20098
#TransformersInBiology #ProteinStructure #AlphaFold #ProtTrans #PLMs #Bioinformatics #DrugDiscovery #PPIprediction #ProteinFunction #ProteinTransformerSurvey

ChemBERTaPolyPharm: Modeling polypharmacy side effects with ChemBERTa and PubMed Encoders
1.This paper presents ChemBERTaPolyPharm, a new deep learning model for predicting polypharmacy side effects by integrating molecular structure embeddings from ChemBERTa with clinical knowledge encoded from PubMed using transformer encoders.
2.The model achieves high accuracy in predicting drug-drug interactions (DDIs), reaching an F1 score of 0.93 across 964 side effect types, outperforming or matching established methods like Decagon, NNPS, and DeepWalk.
3.ChemBERTaPolyPharm innovatively combines SMILES-based chemical structure embeddings and clinical mono-side effect data through a hybrid architecture using both ChemBERTa-77M and PubMedBERT encoders.
4.Each drug is represented as a 979-dimensional feature vector, formed by concatenating ChemBERTa embeddings with PCA-reduced PubMed mono-side effect features; drug pair vectors are aggregated via element-wise summation before DNN classification.
5.The model uses a simple three-layer feedforward neural network with batch normalization and dropout, ensuring regularization while maintaining performance across a wide range of drug pairs.
6.Extensive benchmarking on the TWOSIDES dataset (63,000+ drug pairs) demonstrates the model’s scalability and reliability in real-world scenarios where clinical validation of all combinations is infeasible.
7.Case studies show ChemBERTaPolyPharm outperforms prior models in identifying dangerous interactions, such as those leading to sarcoma or malignant hypertension, supported by literature evidence.
8.While ChemBERTaDDI (the model's predecessor) slightly outperforms in F1 score (0.94 vs. 0.93), ChemBERTaPolyPharm’s dual-encoder design enhances generalization and provides richer input representations.
9.Compared to GCN-based and graph embedding methods like Decagon, ChemBERTaPolyPharm does not rely on complex network topology, making it easier to scale and integrate into broader drug discovery pipelines.
10.The study highlights the importance of combining chemical structure and clinical semantics for accurate DDI prediction, particularly for new or unannotated drug pairs that fall outside traditional databases.
📜Paper: biorxiv.org/content/10.1101/…
#Polypharmacy #DrugSafety #ChemBERTa #DDIPrediction #BiomedicalNLP #TransformerModels #ClinicalAI #Pharmacovigilance

A Machine Learning Pipeline for Molecular Property Prediction using ChemXploreML
1.This work introduces ChemXploreML, a modular desktop application for predicting molecular properties using machine learning, offering flexibility, accessibility, and extensibility for chemists with varying levels of programming expertise.
2.The tool integrates multiple molecular embedding techniques (Mol2Vec and VICGAE) with powerful regression algorithms (GBR, XGBoost, CatBoost, LightGBM) to predict five key properties: melting point, boiling point, vapor pressure, critical temperature, and critical pressure.
3.Mol2Vec delivers slightly better predictive accuracy (up to R² = 0.93 for critical temperature), while VICGAE offers similar performance at a much lower dimensionality (32 vs. 300) and up to 10x faster computation—making it ideal for high-throughput screening.
4.ChemXploreML automates the entire pipeline: from chemical structure preprocessing and embedding generation to model training, hyperparameter optimization (via Optuna), and visualization of results through an intuitive desktop GUI.
5.Datasets were curated from the CRC Handbook of Chemistry and Physics, with SMILES representations canonicalized and cleaned using cleanlab to remove mislabeled or outlier data, ensuring high data quality for training robust models.
6.The system includes advanced visualization tools like UMAP for mapping high-dimensional chemical space, enabling users to explore structure–property trends and cluster molecules based on thermodynamic behavior.
7.Extensive benchmarking reveals that tree-based models paired with Mol2Vec embeddings excel in predicting boiling and critical temperatures, while VICGAE performs competitively—especially for challenging skewed properties like vapor pressure.
8.The modular framework supports future extensions, including ChemBERTa, MoLFormer, GNNs, and classification models, allowing researchers to tailor workflows to specific datasets or property types without code changes.
9.ChemXploreML's core contribution is bridging the gap between high-performance ML and practical chemistry workflows, offering an out-of-the-box platform for molecular design and property prediction in academic and industrial settings.
10.This study also highlights how embedding choice and property distribution shape model performance, with VICGAE emerging as a promising alternative to high-dimensional embeddings for fast, efficient learning on limited data.
💻Code: github.com/aravindhnivas/Che…
📜Paper: arxiv.org/abs/2505.08688v1
#Cheminformatics #MolecularPropertyPrediction #MachineLearning #Mol2Vec #Autoencoder #Regression #MLTools #ComputationalChemistry #UMAP #DataScience

A Survey of Molecular Representation Learning: From Single Modalities to Foundation Models
1.This survey provides a comprehensive overview of molecular representation learning (MRL), tracing its evolution from unimodal string, graph, and image-based encoders to the rise of multimodal and foundation models in chemistry.
2.A central contribution is the categorization of multimodal integration strategies into three types: alignment (contrastive learning), translation (modality-to-modality generation), and fusion (combining features across modalities at various levels).
3.The review emphasizes the limitations of single-modality approaches—such as loss of complementary information, poor generalization, and modality-specific constraints—that motivate the shift toward multimodal frameworks.
4.Multimodal models are shown to leverage complementary data sources like graphs, SMILES, 3D conformers, spectra, and text, enabling more robust and general-purpose chemical representations for diverse downstream tasks.
5.The paper outlines a unified MRL pipeline covering preprocessing, encoding strategies per modality, self-supervised training objectives, and downstream applications like property prediction and molecule generation.
6.It highlights recent encoder architectures: BERT-style transformers for SMILES, GNNs and SE(3) transformers for graphs and 3D structures, and vision transformers for molecular images and videos.
7.Contrastive learning emerges as a leading technique for multimodal alignment, enabling cross-modal representation without needing all modalities at inference time—useful when data is incomplete or noisy.
8.The authors define four criteria for “chemical foundation models”: large-scale diverse data, self-supervised training, high model capacity, and multimodal integration, aiming for robustness under distribution shifts.
9.Recent models like ChemBERTa-2, MolGPS, VideoMol, ChemDFM-X, and MolFM are reviewed as promising steps toward universal chemical foundation models, capable of generalizing across tasks and modalities.
10.The paper concludes with a call to develop large-scale, multimodal, and transferable chemical models that mirror the unification seen in NLP and vision, suggesting this will accelerate scientific discovery across domains.
📜Paper: doi.org/10.26434/chemrxiv-20…
#MolecularRepresentation #Cheminformatics #MultimodalLearning #FoundationModels #ComputationalChemistry #GraphNeuralNetworks #SMILES #SelfSupervisedLearning #DrugDiscovery #AI4Science

LANTERN: TCR-Peptide Binding Prediction via Large Language Model Representations
1. LANTERN is a deep learning framework that leverages large language model embeddings to predict T-cell receptor (TCR) binding to peptide-MHC complexes, achieving high accuracy, especially on novel epitopes in zero-shot and few-shot settings.
2. Unlike traditional models, LANTERN integrates two powerful pretrained encoders: ESM for TCR sequences and MolFormer for SMILES-translated peptides, enhancing its ability to capture structural and chemical properties relevant to TCR recognition.
3. The use of SMILES encoding for peptides is a key innovation, transforming short sequences into chemically informative representations, significantly improving the model’s capacity to distinguish binding from non-binding pairs.
4. LANTERN outperforms both deep learning baselines and other LLM-based models like ChemBERTa, TEINet, and TITAN, achieving an AUC-ROC of 0.88 in realistic negative sampling settings, and showing a 23% performance gap over traditional models.
5. The model demonstrates strong generalization even with minimal data. In few-shot learning experiments, it rapidly learns binding patterns with fewer than 5 examples per peptide, highlighting its potential for applications with limited experimental data.
6. Embedding quality is quantitatively validated: pretraining significantly improves clustering metrics such as Silhouette Index, Calinski-Harabasz Index, Hopkins Score, and Cluster Tendency Score, showing tighter and more meaningful binding/non-binding separation.
7. The model uses hard data splits to ensure peptides in the test set are unseen during training, emphasizing robust generalization rather than memorization—a critical requirement for real-world immunotherapy development.
8. LANTERN’s performance is particularly strong in the more challenging random control setting, and its embedding clusters also reflect biological signal such as epitope frequency, indicating that it learns biologically meaningful features.
9. The framework’s architecture is modular and interpretable, relying on pretrained embeddings followed by a simple MLP classifier—making it adaptable to future improvements in molecular language models or additional data types.
10. LANTERN is a strong step forward for personalized immunotherapy, offering a scalable, data-efficient, and interpretable approach to modeling complex TCR-antigen interactions that are central to vaccine and T-cell therapy development.
💻Code: github.com/qi-c-lab/LANTERN
📜Paper: arxiv.org/abs/2505.01433
#TCR #Immunotherapy #LLM #ProteinLanguageModel #Bioinformatics #MolFormer #ESM #TCell #DeepLearning #Immunology #LANTERN

DeepMolecules: a web server for predicting enzyme and transporter–small molecule interactions
1. DeepMolecules is a user-friendly web server that predicts enzyme– and transporter–small molecule interactions, as well as the kinetic parameters \(k_{cat}\) and \(K_M\), by integrating four state-of-the-art machine learning models into a unified, accessible platform.
2. The server includes ESP and SPOT for substrate prediction (enzymes and transporters, respectively), TurNuP for turnover number (\(k_{cat}\)) estimation, and a graph-based model for Michaelis constant (\(K_M\)) prediction—each optimized for generalizability across wild-type proteins and diverse substrates.
3. DeepMolecules outperforms prior models, leveraging transformer-based protein representations (ESM-1b) and chemical embeddings (ChemBERTa or GNNs) to feed into gradient-boosted decision trees, achieving high accuracy even for previously unseen proteins and metabolites.
4. The enzyme–substrate predictor (ESP) is powered by ProSmith, a multimodal transformer that jointly embeds protein sequences and small molecules—achieving ROC-AUC of 97.2% and accuracy of 94.2%, outperforming previous tools like REME and earlier ESP versions.
5. TurNuP is the only \(k_{cat}\) predictor that uses full chemical reactions as input, employing differential reaction fingerprints (DRFPs) and delivering \(R^2 = 0.44\) on independent test sets, substantially improving over similarity-based methods.
6. The \(K_M\) prediction model uses a graph neural network (GNN) for substrate encoding and performs with state-of-the-art accuracy (\(R^2 = 0.53\)) on diverse enzyme–substrate combinations, including those from uncharacterized species.
7. DeepMolecules also includes SPOT, the first general-purpose transporter–substrate predictor using dual transformer encodings—achieving >92% accuracy and outperforming tools like TooT-SC and TranCEP, which are limited to broad substrate classes.
8. The server supports batch submissions (up to 500 entries per file) and outputs not just prediction scores but curated experimental data from sources like BRENDA, UniProt, and SABIO-RK, with confidence metrics tied to training data frequency.
9. In a use case on human cysteine dioxygenase (CDO1), DeepMolecules correctly identified l-cysteine as a substrate and proposed several novel candidates (e.g., alanine, d-cysteine), guiding focused experimental validation from a pool of 1,400 molecules.
10. DeepMolecules is freely accessible at deepmolecules.org and provides an end-to-end tool for enzyme annotation, biocatalyst design, metabolic modeling, and drug discovery—bridging predictive AI with curated biochemical evidence.
💻Code: DeepMolecules.org
📜Paper: academic.oup.com/nar/advance…
#Enzymes #Transporters #Kinetics #ProteinLigand #DrugDiscovery #MetabolicEngineering #MachineLearning #ComputationalBiology #AI4Science #WebServer #DeepLearning #Bioinformatics

Rethinking the Generalization of Drug Target Affinity Prediction Algorithms via Similarity Aware Evaluation
1. This paper challenges the prevailing evaluation schemes in drug-target affinity (DTA) prediction, showing that conventional random splits produce over-optimistic results by heavily favoring high-similarity samples in the test set.
2. The authors introduce Similarity Aware Evaluation (SAE), a principled framework for creating test splits with user-defined similarity distributions, enabling more realistic and robust assessments of model generalization to novel compounds.
3. In standard random splits, over 88% of test samples are highly similar to the training data. Models like SAM-DTA show sharp performance drops (e.g., R² dropping from 0.65 to -0.64) on low-similarity samples—yet this is hidden in aggregate metrics.
4. SAE formulates test set construction as a constrained optimization problem, using relaxed continuous weights and differentiable approximations of similarity and bin-counting functions, allowing flexible and efficient test set splitting.
5. The authors propose both "balanced splits"—uniform across similarity bins—and "mimic splits" that replicate external test distributions. Both strategies significantly outperform random, scaffold, and SIMPD splits in reflecting real-world generalization.
6. Evaluations across four datasets (EGFR, BACE1, Carbonic anhydrase I/II) and five models (SAM-DTA, PharmHGT, FusionDTA, MolCLR, ChemBERTa) confirm that SAE provides clearer insight into performance degradation across similarity levels.
7. SAE-based splits enable more accurate hyperparameter tuning. Models optimized using mimic splits show improved performance on external datasets compared to those tuned using conventional splits, underscoring its utility in practical deployment.
8. SAE supports additional use cases like generating test sets with predefined maximum similarity (e.g., <0.4 or <0.6), useful for avoiding IP conflicts or targeting novelty in drug screening.
9. The authors open-source their full implementation, providing a PyTorch-based optimizer for test split generation, and demonstrate how SAE can be extended to QSAR, ADMET, protein-protein interaction, and drug-drug interaction prediction tasks.
10. Overall, SAE reframes how generalization in DTA prediction should be evaluated, advocating for similarity-aware testing to ensure models perform reliably when exposed to unfamiliar chemical space.
💻Code: github.com/Amshoreline/SAE
📜Paper: arxiv.org/abs/2504.09481
#drugdiscovery #DTAprediction #machinelearning #QSAR #bioinformatics #generalisability #AIinbiotech #deeplearning #datasetbias #modelvalidation

Pre-trained molecular representations enable antimicrobial discovery @NatureComms
1. This study introduces MolE, a self-supervised framework that learns chemically meaningful molecular embeddings from unlabeled structures, enabling accurate antimicrobial activity prediction even with minimal labeled data.
2. MolE is pre-trained using only 100,000 unlabeled molecules and the Barlow-Twins redundancy reduction objective, creating rich task-agnostic representations that rival or outperform larger models like ChemBERTa and MolFormer.
3. By combining MolE with XGBoost, the authors developed a two-stage prediction pipeline (MolE-XGBoost) capable of estimating antimicrobial potential scores (AP scores) for any compound against 40 microbial strains, including human gut microbes.
4. MolE-XGBoost rediscovered Halicin and Abaucin—two structurally novel antibiotics—entirely from molecular structure alone, validating the model’s ability to generalize beyond known chemical scaffolds.
5. In experimental validation, three out of six predicted compounds (Elvitegravir, Opicapone, and Ebastine) exhibited growth-inhibitory activity against *Staphylococcus aureus*, with Elvitegravir showing strong inhibition at 8 μg/mL.
6. The AP scores correlate strongly (Spearman’s ρ = −0.5) with experimentally measured MIC values, confirming the framework’s ability to prioritize effective compounds and estimate potency.
7. Compared to ECFP4 fingerprints and descriptor-based models, MolE representations capture global and functional group-level features, allowing better discrimination of structural and functional similarity.
8. The approach correctly predicts broad-spectrum vs. narrow-spectrum activity and can rank compounds by predicted effectiveness against Gram-positive or Gram-negative strains using customized AP metrics (K, G, G+, G−).
9. MolE-XGBoost prioritized 158 non-antibiotic compounds with predicted broad-spectrum activity, 53 of which had prior experimental evidence—yielding a ~33% hit rate compared to the 1–3% in traditional screens.
10. The framework is computationally lightweight, requiring far less data and compute than traditional deep learning models, making it accessible for microbiologists with limited computational resources.
11. This work presents a scalable, interpretable, and experimentally validated method for antimicrobial discovery that complements high-throughput screening and opens new directions for drug repurposing and resistance research.
💻Code: github.com/rolayoalarcon/Mol…
📜Paper: nature.com/articles/s41467-0…
#AntimicrobialResistance #DrugDiscovery #MolecularEmbeddings #SelfSupervisedLearning #Bioinformatics #MolE #Antibiotics #MachineLearning #GraphNeuralNetworks #Chemoinformatics #Microbiome

How to generalize machine learning models to both canonical and non-canonical peptides
1. This paper presents a systematic evaluation of machine learning models for predicting peptide functions across both canonical and non-canonical peptides, addressing the challenge of generalization between these two classes.
2. It highlights the limitations of existing methods, which struggle to generalize from canonical peptide data to non-canonical peptides due to differences in structural diversity and limited data availability.
3. The authors introduce a novel approach that utilizes similarity-based dataset partitioning with MAPc and ECFP fingerprints for non-canonical peptides and MMSeqs2 for canonical peptides, optimizing data splits to ensure robust performance.
4. They find that fingerprint-based similarity measures, especially MAPc and ECFP, outperform traditional sequence alignment methods for partitioning canonical datasets, suggesting a shift from conventional practices.
5. To address the problem of representation, the study benchmarks several molecular representation methods, including Chemical Language Models (CLMs) like ChemBERTa-2 and Molformer-XL, protein language models like ESM2 and Prot-T5-XL, and molecular fingerprints like ECFP and PepFuNN.
6. Among all tested representations, general-purpose models like ChemBERTa-2 and Molformer-XL performed better than peptide-specific models such as PeptideCLM and Pepland, with ChemBERTa-2 showing the best generalization across both canonical and non-canonical datasets.
7. The authors demonstrate that training models solely on canonical or non-canonical data results in poor generalization to the other class. However, training on mixed datasets enables successful generalization with minimal performance loss.
8. Jointly trained models effectively generalize to both canonical and non-canonical peptides, even when the dataset is heavily imbalanced, suggesting that representation techniques can effectively bridge the gap between these two peptide classes.
9. The study concludes that ChemBERTa-2 is the most suitable representation technique for joint modeling and recommends further exploration of hybrid approaches combining different representation techniques for better performance.
10. The authors release benchmark datasets and partitioning schemes to aid future research on peptide representation learning.
💻Code: github.com/IBM/AutoPeptideML…
📜Paper: doi.org/10.26434/chemrxiv-20…
#PeptideModeling #MachineLearning #Bioinformatics #CanonicalPeptides #NonCanonicalPeptides #ChemBERTa #Generalization #MolecularRepresentations #PeptideRepresentationLearning

GraphBAN: An inductive graph-based approach for enhanced prediction of compound-protein interactions @NatureComms
- Predicting compound-protein interactions (CPIs) is crucial for drug discovery. GraphBAN introduces a graph-based framework that enhances the prediction of CPIs, particularly for unseen compounds and proteins.
- Unlike traditional methods limited to known compound-protein interactions, GraphBAN excels in inductive predictions, meaning it can predict interactions for entirely new compounds and proteins.
- The model employs a teacher-student learning approach, where the teacher block extracts network structure information, and the student block focuses on node attributes, improving prediction accuracy.
- A domain adaptation module enhances generalization across datasets, making GraphBAN effective even in cross-domain settings where the training and test datasets differ significantly.
- GraphBAN outperforms ten baseline models on five benchmark datasets, demonstrating superior accuracy and robustness in CPI prediction.
- A case study on the Pin1 protein, an important target in cancer therapy, validated GraphBAN’s predictive power. It identified potential inhibitors from a large compound library, showing its real-world applicability.
- The model integrates multiple feature extraction techniques, including Graph Convolutional Networks (GCNs), large language models (LLMs) like ChemBERTa and ESM, and convolutional neural networks (CNNs), ensuring a rich representation of molecular and sequence data.
- Knowledge distillation enables the student model to learn from the teacher’s structured network information, enhancing performance in identifying active compound-protein interactions.
- GraphBAN’s combination of bilinear attention networks (BANs) and conditional domain adversarial networks (CDANs) further refines its predictions, improving generalization to new compound-protein pairs.
- By incorporating advanced feature fusion and graph-based learning, GraphBAN represents a significant advancement in AI-driven drug discovery, helping identify novel drug candidates more efficiently.
💻Code: github.com/HamidHadipour/Gra…
📜Paper: nature.com/articles/s41467-0…
#DrugDiscovery #AI #MachineLearning #GraphNeuralNetworks #Bioinformatics #ComputationalBiology #ProteinInteractions

ChemBERTaDDI: Transforming Drug-Drug Interaction Prediction with Transformers and Clinical Insights
1. ChemBERTaDDI introduces a novel framework combining transformer-based embeddings with clinical data to predict drug-drug interactions (DDIs), addressing the critical issue of adverse reactions in polypharmacy.
2. The framework leverages ChemBERTa-77M-MLM, a transformer pretrained on chemical SMILES strings, to extract rich molecular embeddings. These embeddings are combined with clinical mono side-effect data using dimensionality reduction techniques.
3. A key innovation is the integration of chemical structure and clinical data, enabling ChemBERTaDDI to achieve state-of-the-art performance on benchmark datasets, with an F1-score of 0.94 and AUROC of 0.97.
4. The model demonstrates scalability by handling 964 side effects across 645 drugs, utilizing a unified 979-dimensional feature vector for accurate binary classification of DDIs.
5. ChemBERTaDDI outperforms established models like Decagon and NNPS by combining molecular and clinical insights, proving superior in predicting dangerous polypharmacy side effects such as sarcoma and leukopenia.
6. Ablation studies reveal the critical role of integrating mono side-effect data with chemical embeddings. Models using only chemical features lag behind in predictive accuracy.
7. The methodology includes training a single neural network with frozen ChemBERTa weights, significantly reducing computational overhead while generalizing effectively to new drugs.
8. This approach highlights the importance of transformer models in bioinformatics, pushing the boundaries of drug interaction prediction by incorporating pre-trained molecular representations.
💻Code: github.com/anastasiyagromova…
📜Paper: biorxiv.org/content/10.1101/…
#DrugDiscovery #Transformers #Bioinformatics #DeepLearning #Chemistry

A Chemical Language Model for Molecular Taste Prediction
1. Introducing FART (Flavor Analysis and Recognition Transformer), a chemical language model trained on the largest molecular taste dataset (15,025 compounds), capable of predicting four taste categories (sweet, bitter, sour, umami) simultaneously with over 91% accuracy.
2. FART outperforms previous binary classifiers, delivering state-of-the-art results across individual taste classes, even when benchmarked on larger and more diverse test sets.
3. A key feature is its interpretability: gradient-based visualization highlights molecular substructures driving taste predictions, offering insights into the chemical basis of taste.
4. SMILES augmentation enhances FART’s robustness, ensuring consistency across diverse molecular representations and enabling a confidence metric that boosts prediction reliability to 94%.
5. FART’s architecture, built on ChemBERTa and fine-tuned on taste data, captures nuanced patterns in molecular taste prediction while handling undefined classes, such as tasteless compounds or out-of-distribution molecules.
6. Practical applications include food science, drug formulation, and natural product analysis, providing a tool for rapid, interpretable predictions to accelerate tastant discovery and systematic flavor exploration.
7. By making FART and its dataset publicly available, this study paves the way for further advancements in molecular taste prediction and its integration into broader chemical discovery workflows.
💻Code: github.com/fart-lab/fart.git
📜Paper: doi.org/10.26434/chemrxiv-20…
#ChemicalLanguageModel #TastePrediction #MachineLearning #FoodScience #ChemBERTa #AI

Still processing this incredible moment💫
MoleculeAI won 1st place at #ChemLLmathon2024!
Our "Finetuning ChemBERTa on logS-logP Prediction" project shows the power of merging chemistry & AI. So grateful for my brilliant team who made this possible💖
#chemindix2024

Evaluating the Use of Foundational Chemical Language Models in Multimodal Graph Fusion
1. This study investigates the integration of chemical language models (LMs) with graph neural networks (GNNs) to predict molecular properties, focusing on the potential for improved model performance by leveraging multimodal data fusion.
2. A novel multimodal method is proposed, where pretrained language model embeddings derived from SMILES strings are combined with 3D molecular graph representations. This fusion enriches the GNN’s node features, aiming to capture both textual and spatial data.
3. The authors tested this LM+GNN fusion model on several benchmark datasets, including FreeSolv, ESOL, Lipophilicity (Lipo), QM7, and QM9. Results showed that while the approach enhanced performance on FreeSolv and QM7, it struggled with ESOL, Lipo, and QM9, underperforming compared to a GNN-only model.
4. Interestingly, scaling tests revealed that the LM+GNN model performs relatively better on smaller datasets, suggesting the value of language models may be more pronounced when data is limited. However, as dataset size increases, the advantage diminishes.
5. The study identifies limitations with ChORBERT, the chemical language model used, suggesting it may not be as effective as other models like ChemBERTa-2 on these specific tasks. Differences in pretraining datasets and tokenization approaches might contribute to the variation in model performance.
6. The authors conclude that, while LM embeddings offer some structuring benefits, especially on smaller datasets, 3D graph data is generally more informative for predicting molecular properties. Future work is suggested to develop a model that is pre-trained on 3D graph data to optimize property prediction.
📜Paper: openreview.net/pdf?id=YVnTOm…
#cheminformatics #GNN #AI #MachineLearning #MolecularModeling

SELF-BART: A Transformer-based Molecular Representation Model using SELFIES @IBMResearch
1. SELF-BART introduces an encoder-decoder architecture that leverages SELFIES (SELF-referencing Embedded Strings) to ensure the generation of syntactically valid molecules, outperforming SMILES-based approaches.
2. The model addresses a key limitation in existing molecular models by combining representation learning with auto-regressive generation, making it suitable for both molecular property prediction and molecule generation tasks.
3. SELF-BART’s use of a denoising objective during training improves its ability to learn robust molecular representations, leading to superior performance across nine MoleculeNet benchmark datasets.
4. Compared to graph-based and SMILES-based models like ChemBERTa and MolFormer, SELF-BART achieves state-of-the-art results in classification tasks (e.g., 96.9 ROC-AUC on ClinTox) and regression tasks (e.g., 1.397 RMSE on FreeSolv).
5. The model demonstrates excellent generative capabilities, achieving 99.8% validity, 100% novelty, and high internal diversity in molecule generation tasks, showcasing its potential for drug discovery applications.
6. SELF-BART’s encoder-decoder framework provides the flexibility to generate new molecules from learned representations, allowing researchers to explore novel chemical spaces effectively.
7. Preliminary results indicate SELF-BART’s potential to generate diverse, valid, and novel molecular structures, setting the stage for future studies on conditioned molecular generation.
8. This study underscores the advantages of SELFIES for molecular modeling and highlights SELF-BART’s applicability in both virtual screening and molecular optimization tasks.
📜Paper: arxiv.org/abs/2410.12348

Large language models and their applications in bioinformatics
• Introducing Large Language Models (LLMs) such as BERT and GPT, which have revolutionized bioinformatics by enabling efficient handling of complex biological datasets.
• LLMs, originally designed for natural language tasks, are now tailored for specific bioinformatics applications like protein structure prediction, drug discovery, and sequence analysis, unlocking unprecedented insights.
• A major innovation is the use of transformer architectures in LLMs, which excel in capturing long-range dependencies in DNA, RNA, and protein sequences, significantly improving prediction accuracy for various biological tasks.
• In protein structure prediction, models like ProtGPT-2 and AlphaFold have shown remarkable success in accurately forecasting protein folding and interactions, driving advances in drug discovery and protein engineering.
• LLMs have transformed genomics by tools like DNABERT and RNABERT, which treat DNA and RNA sequences as “languages,” enabling precise identification of gene functions, regulatory elements, and mutations across species.
• In drug discovery, models such as ChemBERTa and SMILES-BERT are enhancing molecular property predictions and bioactivity classifications, providing more efficient paths to identifying drug-target interactions and novel therapeutics.
• The versatility of LLMs extends to gene expression analysis, where models like GeneBERT and scBERT are improving the understanding of regulatory networks and cell-type annotations, accelerating cancer research and precision medicine.
📜Paper: sciencedirect.com/science/ar…

Last Seen Hashtags on Sotwe
Most Popular Users

Elon Musk 
@elonmusk
240.3M followers

Barack Obama 
@barackobama
118.6M followers

Cristiano Ronaldo 
@cristiano
114.4M followers

Donald J. Trump 
@realdonaldtrump
111.8M followers

Narendra Modi 
@narendramodi
106.9M followers

Rihanna 
@rihanna
98.6M followers

NASA 
@nasa
92.2M followers

Justin Bieber 
@justinbieber
91.7M followers

KATY PERRY 
@katyperry
89.9M followers

Taylor Swift 
@taylorswift13
83.9M followers

Lady Gaga 
@ladygaga
75.4M followers

Virat Kohli 
@imvkohli
73.3M followers

Kim Kardashian 
@kimkardashian
70.8M followers

YouTube 
@youtube
68.6M followers

Neymar Jr 
@neymarjr
66.3M followers

Bill Gates 
@billgates
65.2M followers

Selena Gomez 
@selenagomez
63M followers

The Ellen Show
@theellenshow
62.1M followers

CNN 
@cnn
61.7M followers

X 
@x
60.6M followers



