Aaron Wenteler

PhD AI for Drug Discovery @QMUL / @Merck.
Prev. @ {TU Delft, EPFL, Google}.

[Aaron Wenteler @aaronwenteler]

1/21 Excited to introduce Varformer: a multimodal transformer that learns from population-scale missense variation to prioritize drug targets across ancestries, and that scores individual variants along the way. Pre-print, GitHub and model weights at the end of the thread! 🧬🧵

[Aaron Wenteler @aaronwenteler]

@probablybots @kieranrcampbell Love to see it, awesome work !

[Aaron Wenteler @aaronwenteler]

@g_palla1 Thanks Giovanni, means a lot coming from you!

[Aaron Wenteler @aaronwenteler]

1/21 Excited to introduce Varformer: a multimodal transformer that learns from population-scale missense variation to prioritize drug targets across ancestries, and that scores individual variants along the way. Pre-print, GitHub and model weights at the end of the thread! 🧬🧵

[Aaron Wenteler @aaronwenteler]

21/21 And finally, a huge thanks to my co-authors Victor Neduva (@Roche), Wei Wei (@Merck), @ClauPCabrera and @thebarneslab (@QMUL), the Genes & Health Research Team, and the volunteers who took part in Genes & Health!

[Aaron Wenteler @aaronwenteler]

12/21 The top 10 candidates average the 99.0-99.7th percentile and stay above the 98th in every population. The top five all have small-molecule binders, and most of the top 10 have transport or receptor activity, suggesting they belong to the classic drug target families.

[Aaron Wenteler @aaronwenteler]

20/21 Thanks to these amazing projects and resources for sharing their work openly, without which Varformer would not have been possible: @gnomad_project @broadinstitute @OpenTargets @emblebi @GoogleDeepMind @ProteinAtlas & @GTExPortal

[Aaron Wenteler @aaronwenteler]

11/21 Raw scores aren't comparable between populations, so we convert them to percentiles within each population. A target that ranks near the top in all four ancestries has genetic support that doesn't depend on one population. A way to de-risk targets before investing in them.

[Aaron Wenteler @aaronwenteler]

8/21 We tested this on three held-out benchmark sets of increasing difficulty: targets from classic druggable families (GPCRs, kinases, ion channels, nuclear receptors), targets of drugs approved in 2020-2024, and Pharos targets with potent ligands but no approved drug yet.

[Aaron Wenteler @aaronwenteler]

10/21 Varformer beats a logistic regression ablation by 20% and previous state-of-the-art DrugnomeAI by 23% on the hardest (most OOD) benchmark out of the three.

[Aaron Wenteler @aaronwenteler]

9/21 The targets (“positives”) are mixed with highly constrained genes that are unlikely safe (“negatives”). This is distinct from training, where we label targets positive or unlabelled. Varformer reaches a precision@20 of 0.95, 0.86 and 0.48 on the three sets, respectively.

[Aaron Wenteler @aaronwenteler]

7/21 We trained models in four populations: African, Admixed American, Non-Finnish European, and South Asian. Across all cohorts, approved targets consistently cluster within a distinct subspace of the learned embedding space, achieving an ~18-fold enrichment in target density.

[Aaron Wenteler @aaronwenteler]

6/21 Intuitively, this score asks: how much more common is this gene's genetic profile among known targets compared to the genome? Genes similar to successful targets but without a successful drug score high. The prior π only rescales the scores, so the ranking doesn't change.

[Aaron Wenteler @aaronwenteler]

2/21 Drug targets with human genetic support are more likely to succeed in the clinic. Yet most target prioritization methods collapse rich variant information into summary stats. These come disproportionately from European-ancestry cohorts, risking therapeutic inequities.

[Aaron Wenteler @aaronwenteler]

5/21 So what can the model learn given that known targets have a strong selection bias toward those that have been pursued in the past? We prove that our objective is strictly convex, implying that each gene has one optimal score π × density ratio of known targets / all genes.

[Aaron Wenteler @aaronwenteler]

4/21 Target prioritization is naturally a positive-unlabeled problem: few successful targets are known, and a failed trial doesn't necessarily mean a bad target. So we train with a non-negative PU objective on targets that reached Phase III, without explicit negatives.

[Aaron Wenteler @aaronwenteler]

3/21 Varformer learns from the rich variant information itself. A transformer encoder turns the missense variants in each gene into contextual tokens, and cross-attention integrates them with gene-level features such as tractability, safety and tissue expression.