Top Tweets for #multitasklearning
Quantum-Enhanced Multi-Task Learning with Learnable Weighting for Pharmacokinetic and Toxicity Prediction
1. A novel framework, QW-MTL, has been proposed to enhance ADMET (Absorption, Distribution, Metabolism, Excretion, and Toxicity) prediction in drug discovery. This is achieved through a combination of quantum-informed molecular representations and a learnable task weighting mechanism, which significantly improves predictive performance across multiple tasks.
2. QW-MTL integrates quantum chemical descriptors with traditional molecular descriptors to enrich the input features for machine learning models. This hybrid approach provides a more comprehensive representation of molecular properties, leading to better predictions of pharmacokinetic and toxicity profiles.
3. The framework introduces a dynamic task weighting strategy based on sample scale. This allows the model to adaptively balance the contribution of each task during training, addressing the issue of task imbalance and improving overall performance in multi-task learning scenarios.
4. QW-MTL has been evaluated on 13 benchmark tasks from the Therapeutics Data Commons (TDC) and has shown superior performance compared to single-task learning baselines. It achieves state-of-the-art results on multiple tasks, demonstrating the effectiveness of the proposed approach.
5. The model maintains a comparable parameter count to baseline models while significantly improving inference efficiency. This makes QW-MTL a scalable and efficient solution for large-scale molecular screening and property evaluation in drug discovery.
📜Paper: arxiv.org/abs/2509.04601
#QuantumEnhancedLearning #MultiTaskLearning #ADMETPrediction #DrugDiscovery #MachineLearning

Contrastive Multi-Task Learning with Solvent-Aware Augmentation for Drug Discovery
1. This paper introduces a novel pre-training method that incorporates solvent-specific enhancements to molecular conformations, capturing both structural flexibility and environmental context in a unified manner. The model achieves significant improvements in binding affinity prediction, molecular docking, and virtual screening.
2. The framework integrates molecular reconstruction, interatomic distance prediction, and contrastive learning to build solvent-invariant molecular representations. These components work together to improve prediction accuracy and generalizability across different biochemical conditions.
3. The solvent-aware data augmentation pipeline generates ligand conformers under diverse solvent conditions, enabling the model to learn context-dependent interactions and structural diversity. This approach addresses the limitations of conventional models that often neglect solvent effects.
4. The method demonstrates state-of-the-art performance in downstream tasks, including a 3.7% gain in binding affinity prediction, an 82% success rate on the PoseBusters Astex docking benchmarks, and an area under the curve of 97.1% in virtual screening.
5. A case study shows sub-angstrom docking accuracy with a root-mean-square deviation of 0.157 angstroms, highlighting the model’s ability to provide atomic-level insights into binding mechanisms and advance structure-based drug design.
6. The framework supports solvent-aware, multi-task modeling and produces consistent results across benchmarks. It combines binding-pose estimation, affinity prediction, and interaction classification into a single model, reducing the need for manual postprocessing.
💻Code: github.com/1anj/SolvCLIP
📜Paper: arxiv.org/abs/2508.01799
#DrugDiscovery #ProteinLigandInteractions #SolventAwareModeling #ContrastiveLearning #MultiTaskLearning #VirtualScreening

DeepDTAGen: a multitask deep learning framework for drug-target affinity prediction and target-aware drugs generation @NatureComms
1.DeepDTAGen introduces a multitask learning framework that simultaneously predicts drug-target binding affinities and generates target-aware drug molecules using a shared feature space. This dual-task model bridges the predictive and generative arms of drug discovery.
2.Unlike traditional models that tackle drug-target affinity (DTA) prediction and molecule generation separately, DeepDTAGen leverages shared structural and biological features, improving coherence and performance across both tasks.
3.To resolve conflicts that arise in multitask optimization, the authors propose a novel algorithm named FetterGrad. It detects and corrects conflicting gradient directions between the two tasks, leading to more stable and effective joint learning.
4.The model demonstrates superior performance over state-of-the-art models such as GraphDTA, DeepDTA, and GDilatedDTA across three benchmark datasets (KIBA, Davis, BindingDB), achieving lower MSE and higher CI and r²m scores.
5.In DTA prediction, DeepDTAGen shows robustness in drug selectivity analysis and cold-start scenarios, accurately predicting affinities even for unseen drugs and targets, validating its generalizability.
6.The drug generation component is evaluated through both SMILES-based and stochastic strategies. Generated molecules are chemically valid, novel, and exhibit target-specific binding, supported by docking simulations and drug-likeness metrics.
7.Generated drugs maintain polypharmacological properties, effectively interacting with multiple relevant targets. This expands their potential application in multi-target drug design and repurposing scenarios.
8.The authors validate the biological relevance of the learned representations through randomization tests, confirming that performance is not due to chance correlations but genuine learning of drug-target relationships.
9.Chemical and structural analysis reveals that the generated molecules align well with known drug-like properties (e.g., QED, LogP, SAS), and most satisfy Lipinski’s Rule of Five, underscoring their therapeutic feasibility.
10.Ablation studies confirm the importance of key architectural choices such as graph-based drug encoding, Gated-CNN for protein feature extraction, and variational latent space sampling, all of which contribute to performance.
11.Although powerful, DeepDTAGen has limitations: it does not incorporate stereochemistry, nor does it condition generation on physicochemical constraints like QED or LogP directly—highlighting future directions for refinement.
12.Overall, DeepDTAGen represents a meaningful step forward in integrating prediction and generation in drug discovery, providing a flexible, interpretable, and biologically grounded framework.
📜Paper: nature.com/articles/s41467-0…
#DrugDiscovery #AI4Science #MultitaskLearning #Cheminformatics #ProteinDesign

Prot2Token: A Unified Framework for Protein Modeling via Next-Token Prediction
1.Prot2Token introduces a single, autoregressive framework that unifies a wide range of protein modeling tasks—across sequence, residue, and protein-pair levels—by transforming all of them into next-token prediction problems.
2.It leverages task tokens to control prediction types (e.g., structure, function, binding) within a single decoder, enabling prompt-based multitask learning across diverse prediction formats without retraining specialized heads.
3.The approach significantly accelerates structure prediction—up to 1000× faster than AlphaFold2 with MSA—while maintaining competitive accuracy (e.g., TM-score ≈0.54 for Prot2Token vs 0.88 for AF2 w/ MSA).
4.Prot2Token demonstrates strong generalization across 15+ benchmarks, including localization, fluorescence intensity, stability, mutation effect, and protein-ligand binding, using a consistent decoder design.
5.A key innovation is its universal tokenization scheme: classification, regression, and sequence outputs are all converted to standard token sequences, allowing a task-agnostic decoding interface.
6.In regression tasks like fluorescence and stability prediction, Prot2Token outperforms prior methods (e.g., +5.6% in fluorescence Spearman correlation) and benefits from synergistic multitask learning.
7.For binding site prediction, it achieves F1 gains (e.g., 0.63 vs 0.18) when using auxiliary ligand data and self-supervised decoder pre-training to handle spatially sensitive tasks more robustly.
8.Mutation effect prediction with Prot2Token surpasses previous models by a large margin (Spearman 0.9294 vs 0.613 for ProteinNPT), showcasing its capability to learn nuanced sequence-function relationships.
9.In phosphorylation site prediction, Prot2Token beats KinasePhos3 and GPS 6.0 by large margins on all test sets, including rare kinase families (F1: 0.42 vs 0.31 for GPS).
10.Task token embeddings learned by Prot2Token encode meaningful biological relationships, suggesting a potential avenue for interpretable multi-task models and biochemical insight discovery.
11.Despite its broad scope, the current implementation is limited by compute costs—only a subset of all possible tasks were jointly trained. Full multitask scaling remains future work.
12.Output consistency and spatial precision still require architectural or decoding constraints (e.g., enforcing sequence lengths), but decoder pre-training mitigates some limitations.
13.Overall, Prot2Token marks a substantial shift toward general-purpose protein modeling systems—capable of replacing many specialized models—while greatly enhancing throughput and scalability.
💻Code: github.com/mahdip72/prot2tok…
📜Paper: arxiv.org/abs/2505.20589v1
#ProteinDesign #DeepLearning #ProteinStructure #Bioinformatics #PLMs #MultitaskLearning #AlphaFold

Ankh3: Multi-Task Pretraining with Sequence Denoising and Completion Enhances Protein Representations
1.This paper introduces Ankh3, a new generation of protein language models (PLMs) that combines masked language modeling (MLM) with sequence completion tasks during pretraining—enabling richer, more generalizable protein representations without using structural supervision.
2.Ankh3 models outperform prior PLMs like ESM2, ESM3, and earlier Ankh variants on a wide range of downstream tasks including secondary structure prediction, fluorescence regression, GB1 fitness estimation, and residue contact prediction.
3.The core innovation is the adoption of multi-task pretraining using two UL2-inspired objectives: NLU-style denoising with multiple masking rates (15%, 20%, 30%) and S2S-style sequence completion, where the model learns to predict the second half of a protein given the first.
4.This dual-task setup exposes the model to diverse input-output patterns, encouraging it to learn both local residue-level features and global sequence-level dependencies—yielding more robust representations even when trained only on amino acid sequences.
5.Ankh3 is available in two sizes: Ankh3-Large (1.88B parameters) and Ankh3-XL (5.73B parameters), both built on a T5 encoder-decoder architecture with SwiGLU activations, relative attention, and 225 sentinel tokens.
6.Evaluation shows Ankh3-XL achieves top performance among sequence-only PLMs on nearly all tasks. For example, it scores 90.4 Spearman on GB1, 65.4 on fluorescence, and 83.9% precision@L/5 on contact prediction—often outperforming multimodal ESM3 despite not using structural inputs.
7.The benefits of multi-task pretraining are clear: Ankh3-XL outperforms larger models like ProtT5-XXL despite using fewer parameters and no additional data modalities, highlighting the power of architectural and training innovations over raw scale.
8.The model's performance is highly stable across random seeds, unlike ESM3 which exhibited large variance (e.g., up to 8.8% SD in contact prediction)—suggesting better robustness and reproducibility for Ankh3.
9.Interestingly, the best-performing task objective (NLU vs S2S) varied by downstream task: sequence completion (S2S) excelled in regression tasks like GB1 and fluorescence, while NLU worked better for classification tasks like secondary structure.
10.Ankh3 was trained on ~59M sequences from UniRef50 using the JAX-based T5x framework on TPUv4 hardware. Training involved batch sizes of 1024, Adafactor optimizer, and 4M–5M steps depending on model size.
11.All model weights (Ankh3-Large and Ankh3-XL) and training configurations are released on Hugging Face, supporting transparency, reproducibility, and further community-driven research on sequence-only PLMs.
12.Ankh3 demonstrates that thoughtful multi-task design, even with unimodal data, can rival or surpass multimodal models—redefining the limits of protein sequence understanding with elegant and efficient pretraining.
💻Code: huggingface.co/ElnaggarLab/a… huggingface.co/ElnaggarLab/a…
📜Paper: arxiv.org/abs/2505.20052
#ProteinLanguageModels #Ankh3 #MultiTaskLearning #ProteinStructurePrediction #PLMs #Bioinformatics #ComputationalBiology #SequenceModeling #TransformerModels #ESM3 #T5x

Developing a general AI model for integrating diverse genomic modalities and comprehensive genomic knowledge
1.This work presents a general-purpose, multi-task AI model capable of predicting over 20 genomic modalities—including nascent RNAs, histone marks, and chromatin architecture—directly from DNA sequence and ATAC-seq input across diverse human and mouse cell types.
2.The model uniquely integrates epigenomic, transcriptomic, and 3D genome data into a unified framework, accurately predicting tissue-specific enhancer RNAs (eRNAs), gene-level nascent RNA signals, and regulatory interactions even in settings with limited or no direct measurements.
3.In contrast to prior task-specific or sequence-only models like Enformer or Borzoi, this model generalizes across modalities and cell types using ATAC-seq context, enabling robust predictions in unseen tissues and developmental stages.
4.The architecture combines local and global encoders with transformer layers, and includes chromatin interaction-informed self-attention to refine sequence representations—making it effective in modeling long-range genomic interactions.
5.For nascent RNA prediction, the model outperforms average signal baselines across GRO-seq, Bru-seq, and TT-seq data, and successfully captures dynamic transcriptional responses to acute signals like TGFβ-1 treatment in MCF10A cells.
6.Enhancer-level validation using eQTL enrichment shows that the model accurately predicts higher transcriptional activity at genetically important enhancer loci across GTEx tissues, even in the absence of direct eRNA measurements.
7.For non-coding variant function prediction, the model achieves higher accuracy than Enformer and Borzoi in identifying functional eQTLs in unseen cell types, demonstrating superior context specificity and transferability.
8.On lentiMPRA regulatory activity prediction, the model surpasses MPRAnn, MPRALegNet, and ResidualBind in cross-cell type generalization, and accurately recovers active elements across novel cellular contexts.
9.When applied to enhancer–gene regulatory interaction prediction, the model outperforms ENCODE-rE2G using only ATAC-seq input, and further improves performance when integrating its predicted modalities (e.g., histone marks, TF binding, chromatin loops).
10.A mouse-specific version of the model is trained via fine-tuning, achieving state-of-the-art predictions of histone modifications, chromatin contacts (RCMC), and transcription in embryonic and postnatal tissues across multiple modalities.
11.In a mouse inner ear study, the model captures cell type-specific gene expression, enhancer activity, and regulatory loops at the Atoh1 locus—successfully distinguishing active versus silent enhancer usage between hair cells and supporting cells.
12.Overall, this general AI model provides a powerful, biologically grounded framework for cross-species genomic analysis, enabling modality imputation, regulatory interpretation, and enhancer prioritization with minimal experimental input.
💻Code: github.com/liu-bioinfo-lab/g…
📜Paper: biorxiv.org/content/10.1101/…
#Genomics #AI4Science #Epigenomics #RegulatoryGenomics #MultiTaskLearning #GeneRegulation #EnhancerPrediction #Chromatin #ComputationalBiology #CrossSpeciesAI

LEVERAGING LARGE LANGUAGE MODELS FOR ENZYMATIC REACTION PREDICTION AND CHARACTERIZATION
1.This paper systematically evaluates the potential of large language models (LLMs), specifically Llama-3.1 (8B and 70B), for enzymatic reaction modeling across three key biochemical tasks: EC number prediction, forward synthesis (FS), and retrosynthesis (RS).
2.By fine-tuning LLMs using parameter-efficient LoRA adapters, the authors show that multitask learning significantly boosts performance in FS and RS, achieving up to 33.8% and 19.2% match accuracy, respectively—improvements of 7.9% and 5.3% over single-task setups.
3.The 70B model achieves 91.7% macro-average accuracy in EC class prediction (EC1), outperforming zero-shot prompting by over 60%, and approaching SOTA models trained on much larger datasets.
4.Even when trained on just 200 samples, the fine-tuned 70B model achieves 55.3% EC1 accuracy, doubling the zero-shot baseline, and outperforming an XGBoost benchmark in all tasks and data regimes.
5.To improve molecular fidelity, predictions are evaluated using canonical/non-canonical validity checks and Tanimoto similarity. Over 45% of valid outputs in multitask settings had high similarity (Tanimoto > 0.95) to ground truth, indicating useful structural approximations even when predictions are not exact.
6.Multitask fine-tuning showed synergistic benefits, particularly for FS and RS, where shared biochemical knowledge improves generalization and molecular relevance—especially in predicting chemically plausible alternatives.
7.Unlike domain-specific models, the general-purpose LLMs retain versatility across tasks and show transfer effects—models fine-tuned on one task (e.g., EC prediction) improve performance on unseen tasks (e.g., FS or RS) compared to zero-shot prompting.
8.Three LoRA configurations (light, attention, default) are benchmarked. Performance scales with the number of fine-tuned parameters, with full attention and feed-forward tuning (default) consistently delivering best results across both Llama model sizes.
9.Despite computational constraints, the use of LoRA enables scalable fine-tuning with minimal parameter updates (~0.3%), making the method accessible for labs with limited GPU resources.
10.Limitations include possible pretraining data leakage and limited EC subclass diversity in the BRENDA dataset, but the modular design allows easy extension to broader enzymatic datasets or more rigorous held-out evaluation.
11.This work demonstrates that general-purpose LLMs, when properly adapted, can tackle complex biochemical tasks involving enzyme-substrate relationships, opening new paths for AI-guided biocatalysis, reaction discovery, and functional annotation.
💻Code: github.com/Intelligent-molec…
📜Paper: arxiv.org/abs/2505.05616
#Biochemistry #EnzymePrediction #ReactionModeling #LLM #LoRA #Retrosynthesis #SMILES #Bioinformatics #MultitaskLearning #MachineLearning #ICLR2025

Last Seen Hashtags on Sotwe
Most Popular Users

Elon Musk 
@elonmusk
240.2M followers

Barack Obama 
@barackobama
118.6M followers

Cristiano Ronaldo 
@cristiano
114.4M followers

Donald J. Trump 
@realdonaldtrump
111.8M followers

Narendra Modi 
@narendramodi
106.9M followers

Rihanna 
@rihanna
98.6M followers

NASA 
@nasa
92.2M followers

Justin Bieber 
@justinbieber
91.7M followers

KATY PERRY 
@katyperry
89.9M followers

Taylor Swift 
@taylorswift13
83.9M followers

Lady Gaga 
@ladygaga
75.4M followers

Virat Kohli 
@imvkohli
73.3M followers

Kim Kardashian 
@kimkardashian
70.8M followers

YouTube 
@youtube
68.6M followers

Neymar Jr 
@neymarjr
66.3M followers

Bill Gates 
@billgates
65.2M followers

Selena Gomez 
@selenagomez
63M followers

The Ellen Show
@theellenshow
62.1M followers

CNN 
@cnn
61.7M followers

X 
@x
60.6M followers
