Top Tweets for #MoleculeNet
Multi-scale Graph Autoregressive Modeling: Molecular Property Prediction via Next Token Prediction
1. This paper introduces Connection-Aware Motif Sequencing (CamS), a novel graph-to-sequence representation that enables decoder-only Transformers to learn molecular graphs through standard next-token prediction (NTP). CamS bridges the gap between SMILES-based NTP and graph-native masked modeling by serializing molecular graphs into structure-rich causal sequences, achieving state-of-the-art performance on MoleculeNet and MoleculeACE benchmarks.
2. CamS mines data-driven connection-aware motifs and serializes them via scaffold-rooted breadth-first search (BFS) to establish a stable core-to-periphery order. It concatenates sequences from fine to coarse motif scales, allowing the model to condition global scaffolds on dense, uncorrupted local structural evidence. This hierarchical modeling effectively drives attention toward subtle structural edits that determine activity cliffs.
3. The authors instantiate CamS-LLaMA by pre-training a vanilla LLaMA backbone on CamS sequences. Despite relying on minimal priors, CamS-LLaMA outperforms both SMILES-based language models and strong graph baselines. The interpretability analysis confirms that the multi-scale causal serialization improves attention focus on activity-determining structural edits, making the model more sensitive to fine-grained changes in molecular structure.
4. The study demonstrates that CamS resolves the conflict between graph topology preservation and scalable autoregressive training. It provides a unified graph-to-causal-sequence interface that makes molecular graphs directly learnable by standard decoder-only NTP. This approach not only enhances general-purpose representation strength but also boosts discriminability for activity-cliff prediction, where small local motif edits trigger large property shifts.
5. The experimental results show that CamS-LLaMA achieves overall state-of-the-art performance on MoleculeNet and MoleculeACE. On MoleculeNet, it improves average AUROC in classification and average RMSE in regression compared to previous best baselines. On MoleculeACE, it achieves the best average RMSE, outperforming all ML baselines on most tasks. The ablation studies further confirm that multi-scale concatenation is the key driver of performance.
šPaper: arxiv.org/abs/2601.02530v1
#MolecularPropertyPrediction #Transformer #GraphRepresentation #NextTokenPrediction #CamS #MoleculeNet #MoleculeACE

MOLPROPHECY: BRIDGING MEDICINAL CHEMISTS' KNOWLEDGE AND MOLECULAR PRE-TRAINED MODELS VIA A MULTI-MODAL FRAMEWORK
1. A novel human-in-the-loop (HITL) multi-modal framework called MolProphecy has been developed to integrate medicinal chemists' domain knowledge directly into molecular property prediction models. This addresses a critical limitation of existing pre-trained models which often fail to capture the nuanced, interpretive reasoning of human experts.
2. A significant innovation of MolProphecy is its use of ChatGPT as a "virtual chemist." This large language model simulates expert-level reasoning and decision-making, generating dedicated knowledge representations. This synthetic knowledge is then fused with graph-based molecular features using a gated cross-attention mechanism, enabling a synergistic approach to molecular understanding.
3. The framework's design allows for remarkable flexibility: the simulated chemist input from ChatGPT can be seamlessly swapped with insights from real human chemists without requiring any model retraining. This feature makes MolProphecy a highly practical and generalizable solution for collaborative drug discovery workflows.
4. Empirical evaluations on four benchmark MoleculeNet datasets (FreeSolv, BACE, SIDER, and ClinTox) demonstrate that MolProphecy consistently outperforms state-of-the-art models. Notably, it achieved a 15.0% reduction in RMSE on FreeSolv and a 5.39% improvement in AUROC on BACE, showcasing enhanced predictive accuracy.
5. Analysis of the model's interpretability reveals that the integration of chemist knowledge and structural features offers complementary contributions, improving both prediction accuracy and transparency. Fused representations are shown to be more expressive and informative, providing insights that parallel how medicinal chemists combine conceptual reasoning with structural detail.
š»Code: github.com/zhangruochi/MolPrā¦
šPaper: arxiv.org/abs/2507.02932
#ComputationalBiology #DrugDiscovery #MachineLearning #AIinChemistry #MolecularModeling #HumanInTheLoop #LLM #MedicinalChemistry

Multimodal fusion with relational learning for molecular property prediction @CommsChem
1. A new framework, MMFRL (Multimodal Fusion with Relational Learning), has been introduced to significantly enhance molecular property prediction. This approach tackles the challenge of effectively leveraging diverse data sources, or modalities, for molecular analysis.
2. A key innovation of MMFRL is its ability to utilize auxiliary molecular data during pre-training to enrich embeddings, even when these additional modalities are absent during the final inference step. This ensures that valuable information can still contribute to prediction performance.
3. The researchers systematically explored different stages of multimodal fusionāearly, intermediate, and lateāproviding a comprehensive understanding of their respective benefits and trade-offs. This detailed analysis helps in optimizing how various data types are integrated.
4. Empirical results show that MMFRL achieves superior accuracy and robustness compared to current state-of-the-art methods across various molecular property prediction tasks on MoleculeNet benchmarks. The intermediate fusion strategy, in particular, demonstrated strong performance across many tasks.
5. Beyond predictive performance, MMFRL also improves the explainability of learned molecular representations. This offers valuable insights into the underlying chemical properties and aids in practical applications like drug molecule design through post-hoc analysis.
6. Conceptually, MMFRL introduces a modified relational learning metric that provides a more continuous and comprehensive perspective on relationships between molecular instances, moving beyond rigid pairwise similarities. This allows for a deeper understanding of complex molecular interactions.
š»Code: github.com/zhengyjo/MMFRL
šPaper: nature.com/articles/s42004-0ā¦
#ComputationalBiology #DrugDiscovery #MachineLearning #MolecularPropertyPrediction #MultimodalLearning

An open-source family of large encoder-decoder foundation models for chemistry @CommsChem
ļ¼ļ¼Researchers at IBM introduce SMI-TED289M, a family of open-source encoder-decoder foundation models trained on 91 million curated SMILES from PubChem. These models show strong generalization across molecular prediction, reaction yield estimation, and generative tasks.
ļ¼ļ¼SMI-TED289M adopts a transformer-based encoder-decoder architecture with a novel pooling mechanism that allows for SMILES reconstruction while preserving molecular properties. The decoder plays a key role in shaping a chemically meaningful latent space.
ļ¼ļ¼Two versions are available: the base model with 289M parameters, and a Mixture-of-Experts (MoE-OSMI) model with 8Ć289M parameters, enabling scalable performance with task-specific routing and fine-tuning.
ļ¼ļ¼The models outperform or match state-of-the-art baselines across 11 classification and regression benchmarks from MoleculeNet, including QM9, ESOL, FreeSolv, and Tox21, demonstrating impressive accuracy even with minimal fine-tuning.
ļ¼ļ¼In regression tasks, SMI-TED289M achieves the best performance in all five datasets evaluated, including a standout mean absolute error of 1.32 on QM9āsurpassing models like MoLFormer and D-MPNN.
ļ¼ļ¼SMI-TED289M shows robust reconstruction ability on the MOSES benchmark, generating unique and valid molecules with superior scaffold and nearest-neighbor similarity metrics compared to baselines like JT-VAE, LIMO, and CharRNN.
ļ¼ļ¼The model excels in predicting reaction yields for Pd-catalyzed BuchwaldāHartwig CāN cross-couplings, maintaining high accuracy (R² > 0.97) even with just 2.5% of the training data, outperforming models like Yield-BERT and DRFP.
ļ¼ļ¼Latent space analysis reveals SMI-TED289M captures compositionality in molecular structure. Experiments on carbon-chain families confirm the modelās ability to encode meaningful additive relationships with high linearity (R² = 0.99).
ļ¼ļ¼Compared to MoLFormer, SMI-TED289M generates a more chemically interpretable latent space. Nitrogen-containing molecules cluster distinctly with high HOMO energy, validated by a lower DaviesāBouldin index (2.82 vs 4.28).
ļ¼ļ¼ļ¼Few-shot learning tests demonstrate the ability of SMI-TED289M to infer relationships and generate novel molecular structures from just a handful of examples, suggesting its utility in data-scarce chemical design problems.
ļ¼ļ¼ļ¼The model integrates into dZiner, a molecular design agent that conditions generation on target property profiles. SMI-TEDās real-time property inference supports both closed-loop and human-in-the-loop optimization workflows.
ļ¼ļ¼ļ¼Pre-training strategies include separate optimization for encoder token embeddings and decoder reconstruction, with a vocabulary of nearly 3,000 tokens. Training was conducted on 24 GPUs across 40 epochs with a custom RoFormer-based attention mechanism.
ļ¼ļ¼ļ¼MoE-OSMI further improves regression performance by routing samples to specialized sub-models, enhancing scalability without sacrificing generalization, and offering an efficient approach to model expansion.
ļ¼ļ¼ļ¼By combining chemical structure encoding, SMILES reconstruction, and self-supervised learning, SMI-TED289M emerges as a general-purpose foundation model for property prediction, structure generation, and chemical reasoning.
š»Code: github.com/IBM/materials/treā¦
šPaper: nature.com/articles/s42004-0ā¦
#chemistry #moleculardesign #foundationmodels #AI4Science #drugdiscovery #cheminformatics #LLMs #SMILES #openscience

TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondence
ļ¼ļ¼TRIDENT introduces a tri-modal molecular learning framework that jointly models chemical SMILES, natural language descriptions, and hierarchical taxonomic annotations (HTA) to build richer and more interpretable molecular embeddings.
ļ¼ļ¼Unlike most previous models that only use molecular graphs or simple text, TRIDENT integrates 32 taxonomic classification systems to encode nuanced, multi-level functional knowledge for each molecule, significantly improving semantic understanding.
ļ¼ļ¼TRIDENT replaces standard contrastive learning with a geometry-aware volume-based alignment loss. This enables soft, global alignment across three modalities, capturing higher-order interactions rather than just pairwise similarities.
ļ¼ļ¼Beyond global alignment, TRIDENT incorporates a novel local contrastive objective that aligns molecular substructures (like functional groups) with specific sub-spans in text. This fine-grained matching captures substructure-function relationships critical for many tasks.
ļ¼ļ¼To effectively integrate global and local alignment, TRIDENT introduces a dynamic momentum-based mechanism that adjusts their balance during training, ensuring stable convergence and optimal performance across modalities.
ļ¼ļ¼The authors curate a new dataset of 47,269 <SMILES, Text, HTA> triplets from PubChem, enriched using GPT-4o to synthesize high-quality taxonomic descriptions, providing a unique resource for structured semantic molecular learning.
ļ¼ļ¼TRIDENT achieves state-of-the-art performance on 11 downstream property prediction benchmarks, including MoleculeNet and TDC datasets. It consistently outperforms strong baselines like Atomas, MolFM, and MoleculeSTM.
ļ¼ļ¼Ablation studies show that each componentāHTA, local alignment, and volume-based lossāsubstantially contributes to model accuracy. Removing any one leads to significant drops in ROC-AUC across tasks such as Tox21, ToxCast, and BBBP.
ļ¼ļ¼Even on data-scarce tasks like DILI and Skin Reaction, TRIDENT maintains high performance, showing strong generalization ability. This robustness is particularly important for real-world applications in drug discovery and toxicity prediction.
ļ¼ļ¼ļ¼The model architecture is modular: encoders are frozen during training, and only projection layers are optimized. This design allows TRIDENT to remain lightweight while integrating strong prior knowledge from pretrained language models.
ļ¼ļ¼ļ¼TRIDENTās tri-modal, globally and locally aligned representation space opens the door to more interpretable and biologically grounded machine learning in chemistry, potentially enabling better molecular retrieval, classification, and generation.
ļ¼ļ¼ļ¼This work highlights the importance of structured semantic reasoning in molecular modelingāpointing to a future where deep learning systems can understand molecules not just as flat structures, but as entities embedded in biological, ecological, and pharmacological contexts.
šPaper: arxiv.org/abs/2506.21028v1
#AI4Science #DrugDiscovery #MultimodalLearning #MolecularRepresentation #MachineLearning #Bioinformatics #ContrastiveLearning

Leveraging neural network interatomic potentials for a foundation model of chemistry
ļ¼ļ¼HackNIP is a hybrid framework combining pretrained neural interatomic potentials (NIPs) with shallow machine learning models. Instead of end-to-end deep learning, it extracts fixed-length embeddings from NIPs and feeds them into lightweight predictors for structure-to-property tasks.
ļ¼ļ¼This approach achieves state-of-the-art performance on 8 structure-property prediction tasks in the Matbench benchmark, outperforming both feature-based and deep neural network baselines in several cases.
ļ¼ļ¼HackNIP demonstrates exceptional data efficiency. On datasets with fewer than ~10ā“ samples, it often outperforms direct fine-tuning of the same NIP, making it ideal for limited-data regimes common in materials science.
ļ¼ļ¼The model remains competitive with larger end-to-end neural networks even at full dataset scale. On Matbench tasks with over 10ā“ samples, HackNIP shows performance within chemical accuracy and Pareto-optimal error margins.
ļ¼ļ¼HackNIP generalizes well beyond DFT-predicted properties. It achieves strong performance on diverse experimental and ab initio datasets, including Li-ion diffusivity, superconducting transition temperatures, and battery Coulombic efficiency.
ļ¼ļ¼It also shows robust classification performance on molecular-property prediction tasks from MoleculeNet (e.g., BBB penetration, toxicity), with AUC scores comparable to state-of-the-art molecular pretraining models despite using solid-material-trained NIPs.
ļ¼ļ¼A key innovation is identifying which NIP layer embeddings provide optimal features. Early layers often yield better predictions than deeper ones, especially for solid-state tasks, while molecular tasks benefit from deeper layers.
ļ¼ļ¼UMAP visualization and JensenāShannon divergence analyses show that early embeddings retain more task-relevant diversity. Late-layer embeddings become too compressed, losing fine-grained structural information crucial for downstream tasks.
ļ¼ļ¼Compared to end-to-end fine-tuning, HackNIP's simpler regressors undergo more uniform and efficient parameter updates, with less overfitting and better transferability, especially in small data regimes.
ļ¼ļ¼ļ¼This modularity allows users to plug-and-play different NIP backbones and shallow learners. ORB + MODNet yields the best results overall, but other combinations (e.g., MACE + XGBoost) can be effective under different settings.
ļ¼ļ¼ļ¼HackNIP lowers the entry barrier to high-performance materials ML. It is lightweight enough to run on standard hardware, robust across datasets, and suitable for practical use cases in both academia and industry.
ļ¼ļ¼ļ¼This work suggests that hybrid approachesāassigning distinct roles to feature extraction and property mappingācan outperform monolithic models while improving interpretability and reducing computational cost.
š»Code: github.com/parkyjmit/HackNIP
šPaper: arxiv.org/abs/2506.18497v1
#MachineLearning #MaterialsScience #TransferLearning #InteratomicPotentials #ML4Materials #ComputationalMaterials #GraphNeuralNetworks #NLPforScience

Data efficient molecular image representation learning using foundation models
ļ¼ļ¼MoleCLIP is a molecular representation learning (MRL) framework that repurposes OpenAIās vision foundation model CLIP to learn from molecular images. By doing so, it drastically reduces the need for large molecular pretraining datasets while maintaining state-of-the-art performance on key chemical benchmarks.
ļ¼ļ¼The key innovation is using a foundation model trained on generic visual data as the backbone for chemical tasks. MoleCLIP demonstrates that this strategy enables effective generalization, especially under data-scarce scenarios common in chemistry.
ļ¼ļ¼MoleCLIP significantly outperforms prior image-based MRL models like ImageMol and even competes with graph-based models such as GEM on catalysis-related datasets, despite requiring 5ā10Ć less data for pretraining.
ļ¼ļ¼The modelās robustness to distribution shifts is highlighted by its ability to maintain high prediction accuracy even when tested on molecular images containing structural variations never seen during training, such as R-group substitutions.
ļ¼ļ¼MoleCLIP achieves its performance via a stratified training pipeline: CLIP provides general visual features, followed by molecular pretraining (on ChEMBL-25), and then finetuning on small, chemically relevant datasets. This layered approach supports efficient adaptation.
ļ¼ļ¼Pretraining combines two tasks: structural classification using pseudo-labels from MACCS fingerprint clustering, and contrastive learning (via SimCLR), enabling MoleCLIP to form chemically meaningful latent representations from molecular images.
ļ¼ļ¼The model was benchmarked on standard MoleculeNet tasks and four catalysis datasets (DHBDs, NHCs, phosphines for yield and selectivity), achieving strong or superior performance even with as few as 37 examples in some tasks.
ļ¼ļ¼A domain-specific extension, MoleCLIPBV, adds a continued pretraining step using buried volume predictions of phosphines. This further enhances accuracy in catalysis tasks, showing the effectiveness of integrating domain knowledge.
ļ¼ļ¼Image-based inputs offer a path to reuse vast literature data and leverage pretrained vision models. Despite being less compact and flexible than graphs or SMILES, images provide practical advantages when combined with scalable vision backbones like CLIP.
ļ¼ļ¼ļ¼The work suggests a future direction for chemistry machine learning: starting from large general-purpose models and progressively specializing them with increasingly domain-focused datasets for specific molecular properties or reactivities.
š»Code: github.com/Milo-group/MoleCLā¦
šPaper: pubs.rsc.org/en/content/arti⦠#AI4Chemistry
#MolecularRepresentation #DeepLearning #CLIP #Cheminformatics #Catalysis #FoundationModels #TransferLearning

Hydration Free Energies for Small Molecules with Physics-based Descriptors: Graph Neural Network with Cross Attention
ļ¼ļ¼This work introduces a cross-attention-based graph neural network (GNN) to predict hydration free energies (HFEs) of small molecules. The model outperforms several benchmarks while remaining interpretable, with a test MAE of 0.54 kcal/mol on the FreeSolv dataset.
ļ¼ļ¼A key innovation is the use of cross-attention between solute and solvent molecular graphs, enabling the model to capture context-aware soluteāsolvent interactions, which are critical for accurate solvation predictions.
ļ¼ļ¼Unlike most ML models that rely on opaque or high-dimensional features, this framework combines a minimal set of six physically inspired global descriptors with detailed atomic/bond graph features, striking a strong balance between interpretability and accuracy.
ļ¼ļ¼The global features include electrostatic energy (via a closed-form Generalized Born model), polar surface area, hydrogen bond donors/acceptors, logP, and the number of rotatable bondsāall computationally inexpensive and chemically meaningful.
ļ¼ļ¼The GNN architecture integrates message passing with attention mechanisms at both intra- and inter-molecular levels, updating atom, bond, and global embeddings in a physically interpretable fashion.
ļ¼ļ¼A Set2Set readout is used after message passing to derive fixed-size molecular embeddings that are sensitive to graph size and structure, ensuring flexible yet robust representation across diverse molecules.
ļ¼ļ¼Through ablation studies, the electrostatic descriptor and polar surface area emerged as the most influential global features, reinforcing the importance of polarity and soluteāsolvent electrostatics in driving hydration free energy.
ļ¼ļ¼Compared to existing models like MoleculeNet, CIGIN, and MLSolvA, the proposed method reduces MAE by 23ā36%, even rivaling some models using more expensive 3D structural inputs.
ļ¼ļ¼Performance was consistent across nine chemically diverse functional groups in FreeSolv, with most groups achieving MAEs below 0.36 kcal/mol, demonstrating the model's generalizability.
ļ¼ļ¼ļ¼The model is implemented in PyTorch and DGL, trained on FreeSolv with standard 80/10/10 data splits, and exhibits robust performance across five independent runs, confirming statistical reliability.
ļ¼ļ¼ļ¼The study highlights a hybrid modeling approach: combining physics-based insights (GB electrostatics, PSA) with GNN-based structural learning, offering a scalable and explainable tool for solvation modeling in drug discovery and materials science.
ļ¼ļ¼ļ¼Future directions include extending to larger molecules, incorporating conformational sampling, and adapting the framework for broader solvent systems.
šPaper: doi.org/10.26434/chemrxiv-20ā¦
#GraphNeuralNetworks #Solvation #MolecularModeling #MachineLearning #FreeSolv #Cheminformatics #ExplainableAI

Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization
ļ¼ļ¼Mol-LLM is the first generalist molecular language model that robustly integrates both 1D molecular sequences and 2D graph structures across a broad range of tasks, including property prediction, reaction prediction, and molecule generation.
ļ¼ļ¼At the core of its innovation is MolPO (Molecular structure Preference Optimization), a multimodal fine-tuning strategy that teaches the model to distinguish between correct and perturbed molecular graphsāenhancing structure utilization without additional supervision.
ļ¼ļ¼Unlike prior multimodal models that rely on naive next-token prediction, Mol-LLM uses MolPO to optimize structural preferences, significantly boosting performance and generalization, especially on out-of-distribution benchmarks.
ļ¼ļ¼To maximize graph expressivity, Mol-LLM combines two GNN backbonesāGINE and TokenGTāinto a hybrid encoder, with pre-training on both functional group prediction and SELFIES reconstruction for rich molecular representation.
ļ¼ļ¼A Q-Former module bridges the molecular graph and language input, projecting concatenated graph embeddings into a fixed-length token sequence compatible with a Mistral-7B backbone, which is extended with domain-specific tokens for regression, classification, and generation tasks.
ļ¼ļ¼Mol-LLM is trained on a 3.3M instance multimodal instruction-tuning dataset covering five task groups, constructed from sources such as SMolInstruct, Mol-Instructions, and ChEBI-20, enabling flexible handling of diverse molecular queries.
ļ¼ļ¼Across MoleculeNet and other molecular benchmarks, Mol-LLM outperforms state-of-the-art generalist models like LlaSMol, ChemDFM, and 3D-MoLM, notably in reaction prediction and property regression where structural information is key.
ļ¼ļ¼Ablation studies reveal that the addition of MolPO improves both performance and graph discrimination ability across all tasks. GNN pre-training alone also yields consistent gains, especially on tasks requiring recognition of global molecular features.
ļ¼ļ¼Even in text-only settings (e.g., molecule generation or captioning), Mol-LLM matches or exceeds baselines, and its ability to leverage graphs does not impair performance when structural input is absentāhighlighting its multimodal flexibility.
10ļ¼Evaluation on out-of-distribution datasets (AqSol and ORDerly) confirms Mol-LLMās strong generalization, a result attributed to the combination of graph-aware training and broad instruction coverage.
11ļ¼Mol-LLM establishes a new paradigm in molecular LLMs by unifying instruction tuning with graph-aware preference optimization, laying the foundation for future generalist systems in chemical and biomedical domains.
š»Code: github.com/LGAI-Research/Mol⦠šPaper: arxiv.org/abs/2502.02810
#MolLLM #GraphLearning #DrugDiscovery #LLM #MolecularAI #ChemistryAI #MultimodalAI

Synergistic Benefits of Joint Molecule Generation and Property Prediction
ļ¼ļ¼This paper introduces HYFORMER, a novel transformer-based model that jointly learns molecule generation and property prediction from a single set of parameters, using alternating attention mechanisms to enable both autoregressive and bidirectional processing.
ļ¼ļ¼HYFORMER achieves strong performance in both unconditional and conditional molecule generation, and excels in out-of-distribution property prediction and molecular representation learning, offering clear synergistic advantages over separate models.
ļ¼ļ¼The model toggles between tasks using a special [TASK] token, applying causal attention for generation and bidirectional attention for prediction, effectively mitigating gradient interference during joint training.
ļ¼ļ¼In conditional generation on GuacaMol, jointly fine-tuned HYFORMER achieves state-of-the-art control over molecular properties like QED, SA, and logP, outperforming MolGPT and GraphGPT in mean absolute deviation and property variance.
ļ¼ļ¼On the Lo-Hi benchmark for out-of-distribution prediction, HYFORMER surpasses fingerprint-based and deep learning baselines, including ChemProp and Graphformer, in identifying active molecules with novel scaffolds.
ļ¼ļ¼Molecular representation probing on MoleculeNet shows HYFORMER's embeddings outperform those from Uni-Mol, MolGPT, and other joint models like Graph2Seq across linear and KNN evaluation, demonstrating superior transferability.
ļ¼ļ¼HYFORMER maintains competitive performance in unconditional generation, balancing FCD, KL divergence, validity, uniqueness, and noveltyāconsistently placing on the Pareto front for distributional fidelity.
ļ¼ļ¼For property prediction, HYFORMER rivals top-performing models like Uni-Mol and GEM, outperforming other joint models on 8 out of 10 MoleculeNet datasets, confirming its predictive utility despite being a joint model.
ļ¼ļ¼In a real-world use case, HYFORMER is applied to AMP design and outperforms PepCVAE, AMPGAN, HydrAMP, and AMP-Diffusion in generating peptides with high AMP probability and low predicted MIC values.
ļ¼ļ¼ļ¼Attention analysis of the generated peptides reveals that HYFORMER assigns higher weights to amino acids like Arginine and Tryptophan, correlating with known features of antimicrobial activity, offering interpretability.
ļ¼ļ¼ļ¼HYFORMER generates high-fitness peptides under stringent constraints (MIC ⤠2 µM), while maintaining diversity and plausibility, validating its application in targeted peptide discovery.
ļ¼ļ¼ļ¼This work shows that joint training does not compromise individual task performance, but rather enhances cross-task generalization and control, offering a new direction for efficient and unified molecular modeling in drug discovery.
šPaper: arxiv.org/abs/2504.16559
#MoleculeGeneration #PropertyPrediction #DrugDiscovery #Transformers #JointModeling #AMPDesign #HYFORMER #ComputationalChemistry
![BiologyAIDaily's tweet photo. Synergistic Benefits of Joint Molecule Generation and Property Prediction
ļ¼ļ¼This paper introduces HYFORMER, a novel transformer-based model that jointly learns molecule generation and property prediction from a single set of parameters, using alternating attention mechanisms to enable both autoregressive and bidirectional processing.
ļ¼ļ¼HYFORMER achieves strong performance in both unconditional and conditional molecule generation, and excels in out-of-distribution property prediction and molecular representation learning, offering clear synergistic advantages over separate models.
ļ¼ļ¼The model toggles between tasks using a special [TASK] token, applying causal attention for generation and bidirectional attention for prediction, effectively mitigating gradient interference during joint training.
ļ¼ļ¼In conditional generation on GuacaMol, jointly fine-tuned HYFORMER achieves state-of-the-art control over molecular properties like QED, SA, and logP, outperforming MolGPT and GraphGPT in mean absolute deviation and property variance.
ļ¼ļ¼On the Lo-Hi benchmark for out-of-distribution prediction, HYFORMER surpasses fingerprint-based and deep learning baselines, including ChemProp and Graphformer, in identifying active molecules with novel scaffolds.
ļ¼ļ¼Molecular representation probing on MoleculeNet shows HYFORMER's embeddings outperform those from Uni-Mol, MolGPT, and other joint models like Graph2Seq across linear and KNN evaluation, demonstrating superior transferability.
ļ¼ļ¼HYFORMER maintains competitive performance in unconditional generation, balancing FCD, KL divergence, validity, uniqueness, and noveltyāconsistently placing on the Pareto front for distributional fidelity.
ļ¼ļ¼For property prediction, HYFORMER rivals top-performing models like Uni-Mol and GEM, outperforming other joint models on 8 out of 10 MoleculeNet datasets, confirming its predictive utility despite being a joint model.
ļ¼ļ¼In a real-world use case, HYFORMER is applied to AMP design and outperforms PepCVAE, AMPGAN, HydrAMP, and AMP-Diffusion in generating peptides with high AMP probability and low predicted MIC values.
ļ¼ļ¼ļ¼Attention analysis of the generated peptides reveals that HYFORMER assigns higher weights to amino acids like Arginine and Tryptophan, correlating with known features of antimicrobial activity, offering interpretability.
ļ¼ļ¼ļ¼HYFORMER generates high-fitness peptides under stringent constraints (MIC ⤠2 µM), while maintaining diversity and plausibility, validating its application in targeted peptide discovery.
ļ¼ļ¼ļ¼This work shows that joint training does not compromise individual task performance, but rather enhances cross-task generalization and control, offering a new direction for efficient and unified molecular modeling in drug discovery.
šPaper: arxiv.org/abs/2504.16559
#MoleculeGeneration #PropertyPrediction #DrugDiscovery #Transformers #JointModeling #AMPDesign #HYFORMER #ComputationalChemistry](https://pbs.twimg.com/media/Gr9Tr1YXAAE-_lo.png)
AdaptMol: Adaptive Fusion from Sequence String to Topological Structure for Few-shot Drug Discovery
ļ¼ļ¼AdaptMol introduces a few-shot learning framework for molecular property prediction that adaptively fuses SMILES-based semantic features and graph-based topological features, achieving state-of-the-art performance in low-data drug discovery scenarios.
ļ¼ļ¼Its core innovation is the Adaptive Multi-level Attention (AMA) module, which dynamically balances the contributions of sequence and structure at both local (atom/substructure) and global (whole-molecule) levels, ensuring flexible and informative multimodal fusion.
ļ¼ļ¼Unlike prior multimodal methods that rely on naive concatenation, AdaptMol models inter-modality interactions through attention-based weighting, mitigating redundancy and preserving critical pharmacological signals.
ļ¼ļ¼To improve interpretability, AdaptMol integrates a Monte Carlo Tree Search (MCTS) approach for extracting rationalesāchemically valid subgraphs associated with high predicted activityāproviding transparency into its prediction process.
ļ¼ļ¼Across three MoleculeNet datasets (Tox21, SIDER, MUV), AdaptMol surpasses 13 competitive baselines (e.g., MetaGAT, APN, UniMatch, EGNN) under 5-shot and 10-shot settings, improving average ROC-AUC by 4.18% and 2.44%, respectively.
ļ¼ļ¼On the more challenging TDC dataset, which includes tasks across absorption, distribution, metabolism, and toxicity (ADMET), AdaptMol demonstrates strong cross-domain generalization, outperforming MetaGAT and APN in ROC-AUC, F1, and PR-AUC.
ļ¼ļ¼Ablation studies confirm that both local and global attention layers contribute significantly to performance, and that the adaptive weighting mechanism in AMA is essentialāyielding a 3.43% gain over non-adaptive versions.
ļ¼ļ¼Case studies on BACE inhibitors reveal that AdaptMol's rationales highlight biologically relevant substructures like amide and amine groups, aligning with known binding interactions and improving interpretability over traditional GNNs.
ļ¼ļ¼AdaptMol uses prototype-based meta-learning, representing class information through learned molecular prototypes. During inference, new samples are classified by similarity to these prototypes, enabling fast adaptation to unseen tasks.
ļ¼ļ¼ļ¼Its architecture supports flexible plug-and-play use of different encoders (e.g., GCN, GAT, GraphSAGE), and training is stable and efficient using episodic meta-learning on GPU hardware.
ļ¼ļ¼ļ¼Limitations include possible redundancy when applied to simple molecules and the need for more expressive fusion strategies. Future work will explore hierarchical and learnable gating mechanisms to further enhance fusion granularity and adaptability.
š»Code: github.com/DrugDD/AdaptMol
šPaper: arxiv.org/abs/2505.11878
#FewShotLearning #DrugDiscovery #MultimodalFusion #MolecularRepresentation #MetaLearning #ExplainableAI #ComputationalChemistry #AI4Science

Impact of SMILES Notational Inconsistencies on Chemical Language Model Performance
ļ¼ļ¼This study systematically investigates how inconsistencies in SMILES notationāspecifically grammatical and stereochemical variationsāaffect chemical language models (CLMs), with important implications for model performance and reproducibility.
ļ¼ļ¼A survey of 97 CLM studies revealed that 45.4% did not mention SMILES canonicalization, exposing a widespread lack of preprocessing transparency that may hinder reproducibility in cheminformatics.
ļ¼ļ¼Using RDKit-based canonicalization, the authors found that many commonly used datasets contain significant representational variation. For example, 50% of enantiomers and 30% of geometric isomers in MoleculeNet lack full stereochemical annotations.
ļ¼ļ¼The team trained a transformer-based VAE CLM using standardized and randomized SMILES representations and demonstrated that input notation inconsistencies significantly alter latent molecular representations.
ļ¼ļ¼In translation tasksāwhere the model reconstructs canonical SMILES from random SMILESānotation inconsistencies reduced accuracy, especially in datasets with high variation (e.g., ToxCast), underscoring the sensitivity of encoder-decoder architectures to SMILES formatting.
ļ¼ļ¼In contrast, property prediction tasks were largely unaffected by SMILES inconsistencies. XGBoost classifiers trained on latent vectors from raw and standardized SMILES achieved comparable performance, likely due to downstream feature selection mechanisms.
ļ¼ļ¼Visualization via t-SNE and latent distance analysis confirmed that representational discrepancies persist in raw SMILES encodings and propagate into latent space, even though downstream task accuracy remains robust.
ļ¼ļ¼The study further dissected the role of stereochemical information. Removing stereochemistry degraded translation accuracy (from 0.98 to 0.52), while even adding 3D stereochemistry (via RDKit) caused decoding errors, especially in ring closures.
ļ¼ļ¼Token-level error analysis showed that SMILES tokens encoding chirality (e.g., "@", "[") were disproportionately mispredicted, emphasizing the challenge of learning consistent stereochemical features.
ļ¼ļ¼ļ¼Interestingly, models trained entirely on 3D-removed SMILES showed fewer stereochemical errors but made new structural errors (e.g., incorrect ring numbering), suggesting that stereochemistry indirectly supports other structural learning tasks.
ļ¼ļ¼ļ¼The authors recommend adopting consistent SMILES processing practices (e.g., using RDKit with defined canonicalization and isomeric settings) and explicitly reporting them in future CLM studies to ensure reproducibility.
ļ¼ļ¼ļ¼This work provides both empirical benchmarks and practical guidance for the responsible application of CLMs, reinforcing that task-specific preprocessing considerations are essential for reliable molecular generation and analysis.
š»Code: github.com/mizuno-group/Notaā¦
šPaper: arxiv.org/abs/2505.07139
#SMILES #Cheminformatics #ChemicalLanguageModel #CLM #Canonicalization #MoleculeNet #RDKit #MolecularRepresentation #LatentSpace #Stereochemistry #TransformerVAE #Reproducibility #AI4Science

Graph-structured Small Molecule Drug Discovery Through Deep Learning: Progress, Challenges, and Opportunities
ļ¼ļ¼This comprehensive review provides a deep dive into recent advances in graph-based deep learning (DL) techniques for small molecule drug discovery, covering six core tasks: DTI/DTA, DRP, DDI, MPP, MG, and MOāeach critical to modern pharmacology pipelines.
ļ¼ļ¼The authors highlight the increasing dominance of graph neural networks (GNNs) for encoding molecular structure, enabling nuanced understanding of 2D/3D topology and molecular interactionsāfar surpassing traditional SMILES or CNN-based methods.
ļ¼ļ¼Two major drug discovery strategiesātarget-based (DTI/DTA) and phenotype-based (DRP)āare dissected in detail, with comparisons of architectures, datasets, and OOD generalization performance across dozens of methods.
ļ¼ļ¼The review emphasizes the importance of out-of-distribution (OOD) generalization, particularly in drugātarget and drugācell tasks, introducing a standardized taxonomy: Seen-Both, Unseen-Drug, Unseen-Target, and Unseen-Both scenarios.
ļ¼ļ¼For drugātarget interaction prediction, models like PSC-CPI and MGNDTI integrate sequence, graph, and knowledge embeddings to outperform prior baselines in both classification and regression settings.
ļ¼ļ¼In drugācell response prediction, new trends include contrastive learning and domain adaptation (e.g., GraphCDR, CLDR), aimed at addressing variability in omics data like gene expression and mutations.
ļ¼ļ¼DDI models have evolved from SMILES+CNNs to graph-based contrastive and information bottleneck techniques (e.g., IGIB-ISE, PHGL-DDI), enabling better safety prediction and adverse effect modeling.
ļ¼ļ¼In molecular property prediction (MPP), models like Uni-Mol, MolCLR, and GraphMAE use pretraining on large 3D datasets to generalize across quantum, biophysical, and physiological properties with improved sample efficiency.
ļ¼ļ¼The review categorizes molecule generation (MG) methods into property-, structure-, target-, and phenotype-conditioned models, with diffusion models (e.g., GDSS, MOOD) now dominating the field for their flexibility and performance.
ļ¼ļ¼ļ¼In molecular optimization (MO), the field has shifted from latent-space VAEs to search-based and diffusion-guided methods (e.g., DecompOpt, GraphDiT, MARS), allowing controllable and interpretable molecule refinement.
ļ¼ļ¼ļ¼A key contribution is the unified formalism connecting all six tasks and their mathematical representations, offering researchers a roadmap for choosing appropriate techniques, datasets, and models depending on task formulation.
ļ¼ļ¼ļ¼The paper provides one of the most extensive surveys of datasetsāTDC, MoleculeNet, OGB, ZINC, and QM seriesāannotated by task type and sample size, enabling transparent benchmarking and method selection.
ļ¼ļ¼ļ¼Challenges discussed include poor interpretability, generalization to novel combinations, limited adaptability to new experiments, and lack of standardized benchmarksāoffering insights into next-generation research directions.
ļ¼ļ¼ļ¼The authors call for integrating mechanistic knowledge, causal inference, and online learning frameworks to bridge the gap between algorithm development and practical drug discovery applications.
šPaper: arxiv.org/abs/2502.08975
#DrugDiscovery #GraphNeuralNetworks #MolecularGeneration #Bioinformatics #AI4Science #SmallMolecules #MolecularOptimization #ComputationalChemistry #Cheminformatics #DeepLearning

Multi-Modal Molecular Representation Learning via Structure Awareness
ļ¼ļ¼This work presents MMSA, a novel self-supervised framework for molecular representation learning that fuses 2D/3D graphs and molecular images, enhanced by structure-aware hypergraph modeling and memory mechanisms to capture invariant molecular features.
ļ¼ļ¼Unlike existing fusion-based methods that overlook inter-modal interactions and higher-order structure, MMSA introduces hypergraphs and memory banks to preserve both complex dependencies and stable molecular semantics across modalities.
ļ¼ļ¼The multi-modal representation learning module uses autoencoders to embed information from 2D graphs, 3D conformations, and 2D images into a shared latent space, ensuring alignment through contrastive and cross-reconstruction loss.
ļ¼ļ¼MMSA then models higher-order structural relationships via a hypergraph neural network, treating molecules as nodes and constructing hyperedges through k-nearest neighbor similarity in embedding space.
ļ¼ļ¼A memory bank stores invariant structural prototypes as anchors; embeddings are aligned to these anchors to enhance generalization, mitigate overfitting, and preserve critical molecular features across data splits and tasks.
ļ¼ļ¼The entire system is trained with a total loss combining contrastive, reconstruction, memory, and property-prediction objectivesājointly optimizing both the fusion of modalities and structure-aware learning.
ļ¼ļ¼On eight classification benchmarks from MoleculeNet, MMSA achieves state-of-the-art performance, with ROC-AUC gains of 1.8ā10% over prior methods including Mole-BERT, GraphMVP, and MoleculeSDE.
ļ¼ļ¼In regression tasks, MMSA reduces RMSE across ESOL, Lipo, Malaria, and CEP datasets, highlighting its strength in encoding fine-grained molecular properties critical to drug design and material discovery.
ļ¼ļ¼MMSA also shows excellent cross-dataset generalization in molecular retrieval tasks, retrieving chemically similar molecules with high fingerprint alignment, even across datasets with distinct distributions.
ļ¼ļ¼ļ¼An ablation study confirms that both the structure-awareness module and multi-modal loss terms (contrastive, reconstruction) contribute significantly to MMSAās performance, especially when using all three modalities together.
ļ¼ļ¼ļ¼MMSA is highly adaptable, improving the performance of various GNN backbones (GIN, GCN, GAT, GraphSAGE, Graphformer) and can be used as a plugin module to enhance existing graph-based pipelines.
ļ¼ļ¼ļ¼Overall, MMSA establishes a new paradigm for multi-modal molecular representation learning by integrating complementary views and high-order relational priors, with wide applications in QSAR modeling, molecule retrieval, and pretraining.
š»Code: github.com/RUCAIBox/MMSA
šPaper: arxiv.org/abs/2505.05877
#MolecularRepresentation #GraphNeuralNetworks #SelfSupervisedLearning #Hypergraph #DrugDiscovery #MultimodalLearning #MMSA #Bioinformatics #Cheminformatics #MoleculeNet

HIGHER-ORDER MOLECULAR LEARNING: THE CELLULAR TRANSFORMER
ļ¼ļ¼This paper introduces the Cellular Transformer (CT), a topological deep learning (TDL) framework that generalizes graph transformers to operate on cell complexes, enabling the modeling of higher-order molecular structures like rings, fused motifs, and multi-bond systems.
ļ¼ļ¼A key innovation is the augmented molecular cell complex (AMCC), a richer molecular representation where atoms, bonds, and rings are treated as 0-, 1-, and 2-cells, respectivelyāembedding chemical topology directly into the learning architecture.
ļ¼ļ¼CT performs attention not just over nodes or edges but across multiple structural ranks (0D, 1D, 2D) using a novel pairwise and general cellular attention mechanism, capturing multiscale interactions without relying on graph rewiring, virtual nodes, or ad-hoc biases.
ļ¼ļ¼The architecture employs tensor diagrams to formalize attention flow across cochain ranks, integrating both cross-rank and intra-rank attention, guided by neighborhood matrices derived from topological relations like incidence and adjacency.
ļ¼ļ¼To encode structure, CT introduces cellular positional encodings (CPEs), extending Laplacian and random walk encodings to the cellular domain. It also proposes a novel barycentric subdivision encoding (BSPe) that enhances topological locality.
ļ¼ļ¼Extensive benchmarking on MoleculeNet and the Graph Classification Benchmark (GCB) demonstrates that CT consistently outperforms GNNs, MPNNs, and graph transformers, especially in datasets where topological motifs matter most.
ļ¼ļ¼In GCB, CT achieved the highest accuracy (75.4%) compared to other message-passing and kernel-based methods, showing the benefit of high-order attention even in originally graph-based domains.
ļ¼ļ¼On MoleculeNet, CT ranked among the top across both classification (AUC) and regression (RMSE) tasks, performing particularly well in datasets like HIV, ClinTox, and ESOL, where higher-order features are vital.
ļ¼ļ¼The method is highly generalizable: lifting molecular graphs into CCs using tools like TopoX allows CT to apply broadly, even when only graph data is available, making it backward-compatible with existing pipelines.
ļ¼ļ¼ļ¼This work positions CT as a foundation for topologically informed molecular modeling, offering a scalable, interpretable, and efficient alternative to current GNN-based methods, with applications across drug discovery and materials science.
šPaper: openreview.net/pdf?id=GW3h79ā¦
#MolecularModeling #TopologicalDeepLearning #GraphTransformer #DrugDiscovery #CellComplex #MoleculeNet #ICLR2025 #ChemicalML #AttentionMechanism #CellularTransformer

KGG: Knowledge-Guided Graph Self-Supervised Learning to Enhance Molecular Property Predictions
1. This work presents KGG, a self-supervised learning framework that integrates chemical domain knowledgeāspecifically orbital informationāinto graph neural networks, aiming to improve molecular property predictions crucial for drug discovery and materials science.
2. Unlike conventional molecular graph models that overlook orbital interactions, KGG introduces two chemically informed knowledge vectors: atomic hybridization states and bond types (sigma, pi, conjugation). These enrich node and edge features with orbital-level insights.
3. KGG employs a Knowledge Representation Graph (KRG) encoder and a Knowledge Self-Supervised Pre-training (KSSP) module. The KRG captures hierarchical molecular structures (atom, motif, graph levels), while KSSP runs pretext tasks like reconstructing orbital features and adjacency matrices to embed chemical semantics.
4. Using only 250,000 molecules from ZINC15 for pre-trainingāsignificantly fewer than typical modelsāKGG mitigates data contamination risks while achieving strong generalization. This contrasts with models like KPGT, which often rely on millions of pre-training samples.
5. On twelve benchmark datasets from MoleculeNet, KGG consistently outperforms thirteen state-of-the-art self-supervised learning methods for both classification and regression tasks. It improves average ROC-AUC by 2.4% and significantly reduces RMSE and MAE for regression datasets compared to previous best models like HiMol.
6. KGG fingerprintsāgraph-level embeddings extracted from its encoderāoutperform traditional molecular fingerprints (MACCS, ECFP4, RDK7) in both predictive accuracy and data clustering (t-SNE visualizations), highlighting their enhanced representational power.
7. Extensive ablation studies confirm that orbital knowledge, particularly hybridization descriptors, is essential for KGGās superior performance. Removing these features reduces predictive accuracy, validating the synergy between hybridization and bond-type information.
8. KGG also introduces a contamination analysis framework, demonstrating a remarkably low contamination rate (0.2%) during pre-training, compared to 81.7% in models like KPGT, ensuring robust, unbiased performance evaluations.
9. The authors advocate for broader adoption of more robust benchmarking metricsābeyond ROC-AUCāincluding Matthews Correlation Coefficient (MCC) and Average Precision (AP) to better assess performance, particularly in imbalanced molecular datasets.
10. By embedding detailed chemical knowledge directly into graph neural networks, KGG offers a promising path toward more accurate and interpretable molecular property predictions, supporting data-efficient drug discovery and materials design.
š»Code: github.com/ThinhUMP/KGGraph
šPaper: chemrxiv.org/engage/chemrxivā¦
#DrugDiscovery #GraphNeuralNetworks #SelfSupervisedLearning #MolecularRepresentation #Cheminformatics #AI4Science #ComputationalBiology

Unified Molecule Generation and Property Prediction
1. HYFORMER introduces a unified transformer-based model capable of both molecule generation and property prediction, addressing the longstanding challenge of jointly modeling these tasks within a single architecture.
2. This model uses an alternating attention mechanism, seamlessly switching between causal attention (for generation) and bidirectional attention (for prediction), enabling a shared backbone that serves both tasks without performance trade-offs.
3. A key innovation is HYFORMERās unified pre-training strategy, which combines language modeling, masked language modeling, and property prediction losses. This approach balances generative and predictive capabilities within the same model parameters.
4. In unconditional molecule generation, HYFORMER matches or surpasses state-of-the-art models like MolGPT, achieving high validity and diversity on the GuacaMol benchmark while retaining superior FCD and KL divergence metrics.
5. For molecular property prediction, HYFORMER demonstrates competitive performance against top models like Uni-Mol and GEM across the MoleculeNet benchmarks, outperforming other joint models like Graph2Seq on 8 out of 10 datasets.
6. Notably, HYFORMER excels in out-of-distribution property prediction tasks, outperforming models like Chemprop and Graphformer on multiple Lo-Hi benchmark datasets, showcasing its robustness in drug discovery scenarios.
7. HYFORMER also enhances molecular representation learning. In probing experiments, it outperforms both generative and predictive baselines, providing rich, generalizable embeddings that benefit downstream tasks.
8. In a real-world antimicrobial peptide design case, HYFORMER conditionally generates peptides with high success rates (84% for AMPs and 71% for MICs), outperforming specialized models like AMP-Diffusion and HydrAMP, while capturing biologically relevant features.
9. This work highlights the synergistic benefits of joint modeling, offering a scalable and efficient framework for molecular generation and property prediction, with demonstrated success in both benchmarking and practical applications.
š»Code: github.com/IzdebskiAdam/hyfoā¦
šPaper: arxiv.org/abs/2504.16559
#DrugDiscovery #MolecularDesign #TransformerModels #MachineLearning #AI4Science #ComputationalBiology

Data Efficient Molecular Image Representation Learning using Foundation Models
1/ This paper introduces MoleCLIP, a molecular image representation learning framework that leverages the power of foundation models, specifically OpenAI's CLIP, to significantly reduce the amount of molecular data needed for pretraining, while still achieving competitive performance.
2/ MoleCLIP utilizes a visual transformer architecture initialized with CLIPās pretrained weights, enabling it to adapt quickly to chemical tasks, even with smaller datasets. This approach provides an efficient, data-efficient solution to molecular property prediction.
3/ The study highlights how MoleCLIP outperforms traditional models trained exclusively on molecular data, especially in low-data regimes. It was evaluated on MoleculeNet benchmarks and homogeneous catalysis datasets, achieving comparable performance to state-of-the-art models.
4/ One of the key innovations is MoleCLIPās ability to adapt to distribution shifts, thanks to the robust backbone provided by CLIP. This flexibility allows the model to maintain high performance even when tasks or data differ from the pretraining domain.
5/ The use of image-based molecular representations, rather than graphs or SMILES strings, is a unique aspect of MoleCLIP. By transforming molecules into images, it can leverage advancements in vision foundation models, enabling broader applications in chemistry.
6/ MoleCLIPās framework includes pretraining on molecular image data through structural classification and contrastive learning tasks, followed by fine-tuning for property prediction, making it a versatile tool for various molecular property prediction tasks.
7/ The paper demonstrates that MoleCLIPās performance can be further improved by incorporating domain-focused pretraining, as shown by its enhanced prediction accuracy on specific catalysis datasets after additional training on domain-relevant properties.
š»Code: github.com/Milo-group/MoleCLā¦
šPaper: doi.org/10.26434/chemrxiv-20ā¦
#MachineLearning #Chemistry #AI #MolecularRepresentation #DeepLearning #Chemoinformatics #DataEfficiency #FoundationModels #DrugDiscovery

HDBind: Encoding of Molecular Structure with Hyperdimensional Binary Representations @SciReports
1. HDBind introduces hyperdimensional computing (HDC) as a revolutionary approach for screening protein-ligand interactions, offering energy-efficient alternatives to traditional and deep learning-based methods.
2. This work pioneers the use of HDC with graph-based encodings for molecular data, significantly outperforming existing methods on datasets like MoleculeNet and LIT-PCBA.
3. Key innovation: HDC enables fast, low-energy computations without the need for gradient-based optimization, demonstrating up to an order-of-magnitude improvement in energy efficiency compared to neural networks.
4. It utilizes novel encoding strategies, including random projections and molecular graph-based representations, to create ultra-lightweight models that maintain accuracy while reducing computational costs.
5. HDBind is benchmarked on real-world drug discovery datasets, where it outperforms traditional molecular docking and competes strongly with advanced neural networks in predicting binding affinities.
6. This method sets a new standard for integrating lightweight computational frameworks into drug discovery pipelines, promising scalable and sustainable virtual screening applications.
š»Code: github.com/LLNL/hdbind
šPaper: nature.com/articles/s41598-0ā¦
#DrugDiscovery #HyperdimensionalComputing #Bioinformatics #MolecularModeling #EnergyEfficiency

Multi-View Mixture-of-Experts for Predicting Molecular Properties Using SMILES, SELFIES, and Graph-Based Representations
1. This study presents MoL-MoE, a Multi-View Mixture-of-Experts framework designed to predict molecular properties by integrating latent representations from SMILES, SELFIES, and molecular graphs. This multi-view approach leverages the unique strengths of each molecular representation, enhancing prediction accuracy across diverse molecular tasks.
2. MoL-MoE utilizes a gating network to selectively activate experts based on task requirements, dynamically adapting to the input. With 12 experts (4 for each modality), this adaptive mechanism provides flexibility and optimizes the modelās focus, improving both classification and regression performance on MoleculeNet benchmarks.
3. Experimental results demonstrate MoL-MoEās superiority over state-of-the-art models across nine datasets, including complex multi-task classification tasks like Tox21 and SIDER. The model achieves top results with a routing setting of k = 6, consistently outperforming existing methods by dynamically selecting relevant expert networks.
4. Analysis of expert activation patterns reveals that MoL-MoEās gating network tends to prioritize SMILES and SELFIES representations for certain tasks, while other tasks benefit from a balanced use of all three representations. This adaptability indicates MoL-MoEās capability to capture distinct molecular features, tailoring predictions to the unique demands of each dataset.
5. MoL-MoE achieves robust results on regression tasks, particularly in predicting properties like solubility and lipophilicity. Its balanced approach across multiple molecular views allows it to generalize effectively, offering a comprehensive tool for molecular property prediction that adapts to the intricacies of diverse chemical data.
šPaper: openreview.net/pdf?id=Mifpmeā¦
#MolecularPropertyPrediction #MachineLearning #Chemoinformatics #MultiViewLearning #MixtureOfExperts #SMILES #SELFIES #GraphRepresentation

Pre-trained Molecular Language Models with Random Functional Group Masking
⢠This paper introduces MLM-FG, a SMILES-based molecular language model that employs random masking of functional groups to enhance molecular property prediction. Unlike traditional models that mask random subsequences, MLM-FG focuses on key chemical substructures, leading to a deeper understanding of molecular features.
⢠MLM-FGās functional group-aware masking strategy allows it to infer structural information implicitly, outperforming both SMILES- and graph-based models, including those with explicit 3D information, on 9 out of 11 benchmark tasks.
⢠Utilizing a transformer-based architecture, MLM-FG captures molecular representations by encoding functional groups, making it exceptionally effective in tasks like drug discovery, where structural nuances are critical.
⢠Experiments on MoleculeNet benchmarks, covering classification and regression tasks, show MLM-FG achieves superior predictive performance over leading models such as MoLFormer and GEM, demonstrating significant gains in prediction accuracy.
⢠The modelās capability to learn structural relationships directly from SMILES strings, without 3D data, highlights its potential to improve molecular analysis and reduce data annotation costs, making it suitable for large-scale applications.
š»Code: anonymous.4open.science/r/MLā¦
šPaper: arxiv.org/abs/2411.01401
#MolecularLanguageModel #FunctionalGroup #SMILES #DrugDiscovery #Bioinformatics

A Probability Contrastive Learning Framework for 3D Molecular Representation Learning
⢠This study proposes a probability-based contrastive learning framework designed to tackle false positive and false negative pair issues in 3D molecular representation learning, enhancing accuracy in molecular property prediction.
⢠Unlike conventional methods like MolCLR, which rely on fixed augmentations that can mislabel structurally similar molecules as negatives, this framework introduces learnable weights for data pairs, dynamically adjusted using Bayesian inference.
⢠The model optimizes representation quality by applying a stochastic expectation-maximization (EM) algorithm, which iteratively refines data pair weights and model parameters.
⢠Experimental results show state-of-the-art performance across 13 of 15 property prediction tasks on MoleculeNet and 8 of 12 on QM9, demonstrating improved downstream effectiveness.
⢠By reducing reliance on rigid augmentation heuristics, this method adapts well to both 2D and 3D datasets, enhancing robustness in drug design and molecular property predictions.
⢠The probabilistic frameworkās flexibility allows integration with existing models, including Uni-Mol and Equiformer, showing marked improvements over traditional contrastive and non-contrastive learning approaches.
šPaper: jingchensun.github.io/papersā¦
#MolecularRepresentation #ContrastiveLearning #MachineLearning #Bioinformatics #DrugDesign #AIforChemistry

Contextual Representation Anchor Network to Alleviate Selection Bias in Few-Shot Drug Discovery
1. The study introduces the Contextual Representation Anchor Network (CRA), a new approach designed to mitigate selection bias in few-shot drug discovery, where the scarcity of labeled data is a major issue.
2. CRA leverages contextual representation anchors, which act as class-level bridges between labeled and unlabeled data. This dual-augmentation mechanism helps capture richer contextual knowledge, enhancing molecular feature representations.
3. CRA introduces context augmentation, which dynamically retrieves similar unlabeled molecules to refine class-level anchors, thereby enriching the task-specific contextual knowledge that helps mitigate sample selection bias.
4. Anchor augmentation further transfers this enriched knowledge into molecular representations, resulting in improved intra-class feature consistency and enhanced inter-class distinction.
5. CRA significantly outperformed state-of-the-art models on the MoleculeNet and FS-Mol benchmarks, with average improvements of 2.60% (AUC) and 3.28% (āAUC-PR). It also demonstrated superior generalization in domain transfer experiments, achieving notable gains in AUC and āAUC-PR.
6. The study emphasizes that mitigating selection bias in few-shot learning is crucial for accurate molecular property predictions, especially in the highly selective and biased environments of drug discovery.
šPaper: arxiv.org/abs/2410.20711
#FewShotLearning #DrugDiscovery #MachineLearning #Bioinformatics #DeepLearning #ContextualRepresentation #SelectionBias #CRA #MolecularPropertyPrediction

GDMol: Generative Double-Masking Self-Supervised Learning for Molecular Property Prediction
⢠This paper presents GDMol, a novel molecular property prediction framework that leverages generative double-masking within a self-supervised learning setting.
⢠GDMol introduces a unique two-stage masking approach, first on molecular graphs and then on latent feature vectors, allowing the model to capture both local and global molecular information.
⢠The model significantly outperforms state-of-the-art methods across multiple benchmarks, including datasets from MoleculeNet like BBBP, BACE, HIV, Tox21, and SIDER.
⢠GDMol employs advanced graph neural networks (GNNs), such as TransformerConv and GINEConv, to extract meaningful molecular embeddings for improved prediction.
⢠A critical innovation is the interpretability of GDMol, which uses gradient changes to reveal which atomic features contribute positively or negatively to predictions, guiding molecular design.
⢠Experimental results highlight that GDMol achieves the highest ROC-AUC scores on single classification tasks such as BBBP and BACE, improving by 5% over competing models.
⢠The use of dual masking for pre-training ensures better feature extraction, allowing for more robust molecular representations, even with limited labeled data.
⢠GDMol demonstrates practical value by identifying critical chemical substructures with positive or negative contributions, aiding drug optimization and compound improvement.
š»Code: github.com/LetuQingge/JCB-24ā¦
šPaper: doi.org/10.1002/minf.2024001ā¦

SELF-BART: A Transformer-based Molecular Representation Model using SELFIES @IBMResearch
1. SELF-BART introduces an encoder-decoder architecture that leverages SELFIES (SELF-referencing Embedded Strings) to ensure the generation of syntactically valid molecules, outperforming SMILES-based approaches.
2. The model addresses a key limitation in existing molecular models by combining representation learning with auto-regressive generation, making it suitable for both molecular property prediction and molecule generation tasks.
3. SELF-BARTās use of a denoising objective during training improves its ability to learn robust molecular representations, leading to superior performance across nine MoleculeNet benchmark datasets.
4. Compared to graph-based and SMILES-based models like ChemBERTa and MolFormer, SELF-BART achieves state-of-the-art results in classification tasks (e.g., 96.9 ROC-AUC on ClinTox) and regression tasks (e.g., 1.397 RMSE on FreeSolv).
5. The model demonstrates excellent generative capabilities, achieving 99.8% validity, 100% novelty, and high internal diversity in molecule generation tasks, showcasing its potential for drug discovery applications.
6. SELF-BARTās encoder-decoder framework provides the flexibility to generate new molecules from learned representations, allowing researchers to explore novel chemical spaces effectively.
7. Preliminary results indicate SELF-BARTās potential to generate diverse, valid, and novel molecular structures, setting the stage for future studies on conditioned molecular generation.
8. This study underscores the advantages of SELFIES for molecular modeling and highlights SELF-BARTās applicability in both virtual screening and molecular optimization tasks.
šPaper: arxiv.org/abs/2410.12348

Last Seen Hashtags on Sotwe
Most Popular Users

Elon Musk 
@elonmusk
240.2M followers

Barack Obama 
@barackobama
118.6M followers

Cristiano Ronaldo 
@cristiano
114.4M followers

Donald J. Trump 
@realdonaldtrump
111.8M followers

Narendra Modi 
@narendramodi
106.9M followers

Rihanna 
@rihanna
98.6M followers

NASA 
@nasa
92.2M followers

Justin Bieber 
@justinbieber
91.7M followers

KATY PERRY 
@katyperry
89.9M followers

Taylor Swift 
@taylorswift13
83.9M followers

Lady Gaga 
@ladygaga
75.4M followers

Virat Kohli 
@imvkohli
73.3M followers

Kim Kardashian 
@kimkardashian
70.8M followers

YouTube 
@youtube
68.6M followers

Neymar Jr 
@neymarjr
66.3M followers

Bill Gates 
@billgates
65.2M followers

Selena Gomez 
@selenagomez
63M followers

The Ellen Show
@theellenshow
62.1M followers

CNN 
@cnn
61.7M followers

X 
@x
60.6M followers
