Can LLMs simulate biology?
We tested frontier models on real lab experiments: predict how cells respond to drugs, gene knockdowns, and immune signals. Even the best get only part of the response right. Lots of room to improve.
Introducing BioSimBench, built on @NVIDIAAI. 🧵
Can LLMs simulate biology?
We tested frontier models on real lab experiments: predict how cells respond to drugs, gene knockdowns, and immune signals. Even the best get only part of the response right. Lots of room to improve.
Introducing BioSimBench, built on @NVIDIAAI. 🧵
Why simulate? Drugs, genes, and cell contexts create far more experiments than any lab can run. A co-scientist reasons about mechanisms. A simulator answers a different question: in this cell, after this intervention, what will the experiment measure?
Language models can predict how cells respond to perturbations and adapt to new cell contexts with few measurements. On two new cell lines, CT-Perturb reached 82% of its full-data score using just 10% of the measurements.
https://t.co/hjfblpv0Co
Rat and human liver cells do not respond to a drug in exactly the same way. On five held-out compounds, CT-Tox predicted human gene-expression responses with about one-sixth the error of copying across matching rat genes.
https://t.co/ASRoKfpIBL
This is pretty cool. Anthropic and BigHat used Claude to design antibody variants, then made and tested them in the lab. Along the way, they went back to the raw data and changed how they measured success.
https://t.co/4xTapfkM4e
Can preclinical data help us predict how patient cells respond to drugs?
We trained CT-Translate with and without preclinical data. Adding it improved drug-response predictions for patient-derived cells across 27 of 38 compounds.
https://t.co/wGAvf0sV1G
Can reinforcement learning improve predictions of how cells respond to drugs?
In the C2S-Scale research behind CT-Perturb, rewarding agreement with measured gene programs improved held-out predictions for interferon response and apoptosis.
https://t.co/hjfblpv0Co
How much does a rat liver response tell us about human cells?
Across 26 held-out compounds, CT-Tox achieved 0.74 rank agreement with measured human Hallmark pathway responses, versus 0.35 for copying rat responses to human genes.
https://t.co/ASRoKfpIBL
Which patients will respond to immunotherapy?
Across 5 cohorts, 43% responded. In CT-Translate’s top-ranked third: 61%.
451 patients. Every prediction made by a model that never saw that patient.
Retrospective results:
https://t.co/wGAvf0sV1G
It runs on-prem, frozen and versioned on a single GPU, no external calls. Your fine-tuned weights stay on your premises and run on the licensed CellType foundation model.
Built with a top-10 pharma translational team. Test it on patients you hold out:
https://t.co/yxrw9vyMY8
Predicting patient outcomes from lab data is a long-sought goal in drug development.
Introducing CT-Translate, a preclinical-to-clinical foundation model built to do it.
In its top-ranked third of immunotherapy patients, 61% responded vs 43% overall.
https://t.co/yxrw9vyf8A
One model, 69 clinical questions. In: a tumor profile, clinical details and a plain-English question. Out: a probability per patient, so patients can be ranked.
Built on CT1, our foundation model. Trained on drug screens and more than 13,000 patients across 20 studies.