Rohit Satija

co-founder | swe @StanfordMed

[Rohit Satija @rohitium]

"All bio foundation models lead to target prioritization"

[Patrick Kidger @PatrickKidger]

So Anthropic is doing drug design, and my feed is now full of jaded bio folks declaring what a terrible idea this is.

"Drug design is hard" / "don't compete with your customers" / "if they were serious they'd be acquiring Cradle, Boltz" / etc

I disagree + I am SO EXCITED.

1/2

[Rohit Satija @rohitium]

The point I am trying to make is that learning an energy landscape from empirical cellular imaging trajectories, a pre-requisite for training an accurate model for cell state dynamics, is bound to break the assumption of Markovianity. In other words, your energy function(al) will have tiny errors due to experimental resolution and/or artifacts of your apparatus which will inhibit adequate replication of the true dynamics.

This is the same issue that has plagued the MD community for several decades now. You could argue that Alphafold is a counter example but you need to scratch below the surface only a little bit to realize why protein structure prediction does not help with inferring protein motion.

Here are a few practical model-agnostic Markovianity tests you could try to see the light:
1. Rodríguez‐Girondo, M., & de Uña‐Álvarez, J. (2012). A nonparametric test for Markovianity in the illness‐death model. Statistics in Medicine, 31(30), 4416-4427.
2. Berezhkovskii, A. M., & Makarov, D. E. (2018). Single-molecule test for Markovianity of the dynamics along a reaction coordinate. The journal of physical chemistry letters, 9(9), 2190-2195.
3. Willareth, L., Sokolov, I. M., Roichman, Y., & Lindner, B. (2017). Generalized fluctuation-dissipation theorem as a test of the Markovianity of a system. Europhysics Letters, 118(2), 20001.

[Rohit Satija @rohitium]

I generally like this person's takes, but there are simple tests in the theoretical biophysics literature to test whether a trajectory is Markovian.

My strong suspicion is this cell state trajectory will fail those tests and this whole virtual cell argument will fall apart

[Parmita Mishra @parmita]

so what is “state”?

state is the information you would need to predict the system’s next move.

[Rohit Satija @rohitium]

Link to paper: https://t.co/Rfj0Gm58d0

[Rohit Satija @rohitium]

For this task, we found that RAG had similar performance as QSP (if not worse). A simple way to understand this result is that RAG replaces the full text article with a set of evidence snippets that may or may not contain the information (e.g. sequences, genes, drugs, etc) we are interested in. Since many articles do not contain the correct answer, RAG is more susceptible to false positives than base models with access to full text articles.

[Rohit Satija @rohitium]

This work was done in collaboration with Robert Shafer, Chiara Sabatti, Kaiming Tao, and other amazing researchers at @Stanford.

[Rohit Satija @rohitium]

Finetuning (FT) on an expert-curated instruction set improved performance across the board for GPT-4o; but for the Llama3.1 models, finetuning did not improve recall. For Llama3.1-70B, finetuning improved accuracy, precision, and F1 reaching statistical significance, while for Llama3.1-8B, finetuning only improved precision.

Interestingly, a question-specific prompting (QSP) approach, analogous to the one used by Cao et al Annals of Internal Medicine 2025 for systematic reviews, had a complementary effect. For every model, this approach improved recall without any finetuning, with increasing improvements as model size/capability decreased! Unfortunately, these complementary benefits of finetuning and prompting were not additive – FT+QSP did not improved performance over FT.

[Rohit Satija @rohitium]

To start, we recapitulate the performance improvement as model size/capability increases: GPT-4o > Llama3.1-70B > Llama3.1-8B. However, vanilla models generally found it harder to get good recall, i.e. they predicted more false negatives than false positives.

[Rohit Satija @rohitium]

As tokenmaxxing frontier models becomes increasingly infeasible for regular research, exploiting abilities of smaller/open source models becomes an attractive option. We set out to benchmark LLM performance on a practical problem of extracting structured information from full-text biomedical research papers to populate the Stanford HIV Drug Resistance database.

[Rohit Satija @rohitium]

Can finetuning LLMs outperform RAG or advanced prompting techniques for a niche biomedical knowledge-work use case? (Link to paper in comments)

[Rohit Satija @rohitium]

@SGRodriques Software engineering isn't going anywhere either. Case in point, look at anthropic's purchase of Stainless

[Rohit Satija @rohitium]

This is such an amazing story and finally provides a clear path for personalized medicine 🖤

[Séb Krier @sebkrier]

This is wild. https://t.co/fA4oTX8fB9

[Rohit Satija @rohitium]

I asked codex to solve all my problems and it gave me the single cell expression levels of all protein coding genes in my body

[Tanishq, Ph.D. at ICML @iScienceLuvr]

Google did a prospective clinical study of their AMIE medical LLM chatbot in the clinic!

They used AIME to conduct clinical history taking and present potential diagnoses for patients to discuss with their provider at urgent care appointments at Beth Israel Deaconess Medical Center.

Blinded assessment of AMIE and primary care provider differential diagnosis (DDx) and management (Mx) plans suggested similar overall DDx and Mx plan quality, without significant differences for DDx (p = 0.6) and appropriateness and safety of Mx (p = 0.1 and 1.0, respectively).

"PCPs outperformed AMIE in the practicality (p = 0.003) and cost effectiveness (p = 0.004) of Mx. "

[Rohit Satija @rohitium]

Larger 2-state folders fold faster! https://t.co/PrUg8nlTSS

[Rohit Satija @rohitium]

There's something very chilling about this coming from an authoritative figure like Demis. Why do a small group of people get to decide the balance of benefit vs harm for humanity?

[Tree Climber @techtreeclimb]

@AdamZweiger Demis did get asked about this in an April 2025 interview with Time

[Rohit Satija @rohitium]

“Even though you do everything right you don’t always get what you hope for”

One of my favorite shows lately (and definitely one of the only live action shows I like) is The Pitt. In one of the episodes, Dennis Whitaker (4th year med student) loses a patient. He blames himself for his death - patient had unstable angina due to CAD masquerading as GI pain from gallstones

The attending (Dr Robby) reassures him: "We did an EKG. We did a troponin. He had a HEART score of three" → "No doctor on the planet could have caught this." My doctor friends love this show because it provides a realistic version of the problems they deal with at work.

Inspired by it, I created this game (https://t.co/LQpnarDKGc) that uses transcripts from the show to simulate an emergency room - you pick a case from one of the episodes, then make decisions to inquire patient history, hypothesize differential diagnoses, order labs, prescriptions, perform procedures, and make a final disposition.

It uses a senior attending (AI Dr Robby) that provides hints based on your state in the game, and a final debrief to help you understand how you did. Check it out!

[Marc Finzi @m_finzi]

1/🧵 We are very excited to release our new paper! From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence
https://t.co/M8ETQk9gHz
with amazing team @ShikaiQiu @yidingjiang @Pavel_Izmailov @zicokolter @andrewgwils