youssef

Building agents for clinical development @panoptivehealth

Prev post-training eval team at @apple

[youssef @ymkallel]

@soleilavawizman really thorough writeup!

[youssef @ymkallel]

Make every biopharma ai-native

[Professor Oak @Prof_Oak_]

Surprise 1: Anthropic acquired a one-person biotech
.
.
.
Surprise 2: that person is me
Surprise 3: I’m here to tell you all about it

Yes, this is what you think it is; after 6 yrs of increasingly loose anonymity: the Professor Oak reveal post

…and yes, it has gotten so boring around here that I’m literally doxxing myself to spice up the timeline

-------
Ok, allow me to introduce myself. My name is Cody Tranbarger. I mostly tweet, but occasionally find time to dabble in drug dev & starting/building biotechs. My past lives span the VC, crossover, startup continuum; Prof Oak has been my way of keeping a digital foot in those worlds, despite having my physical feet planted elsewhere.

The past 18 mo, ‘elsewhere’ was Mithril Medicine, a co. I founded to develop bsAbs for LAMA2-CMD & other ECM-related genetic diseases.

Mithril was born out of hands-on building with @MikeNGladstone, @JasonPaulRhodes, @LifeSciVC, & team @atlasventure. Somehow, I made enough half-decent calls along the way (even a broken clock is right twice a day, eh?) to convince them Mithril/I warranted a small seed. While I can hardly rationalize such an outrageously cavalier bet, I’m grateful for it, & I'd partner with Atlas again in a heartbeat.

In typical AV fashion, Mithril’s primordial form was virtual, decentralized, & ultra-lean. This stage tends to be short-lived, but for Mithril, it worked surprisingly well, for surprisingly long, thanks in part to the rapid maturation of the AI tools I used liberally.

Eventually, Mithril became a bit of an experiment itself - to find the upper bound of how far it could scale as a single operator + a bunch of Claude (+ a stellar bench of part-time consultants).

In parallel, I started spending time with @nc_frey, @an1lam, & @CoefficientBio team; and promptly experienced, for the 1st time, AI at the frontier. Soon thereafter, Mithril stopped looking like an experiment, & started looking like a preview of an entirely new operating model.

From that spark came “Pipeline-in-a-Person.” The idea immediately resonated when we shared it here back in April; and it was central to the shared vision that led to Anthropic’s acquisition of Mithril, & my joining the team.

Why join Anthropic? Three reasons.
(1) For Mithril, the programs continue, with more resources & infrastructure behind them, and as much Claude as I can put to work.

(2) For me, the chance to start several more programs, & find out how many one person can actually run in parallel - an almost unprecedented luxury for a curious drug hunter.

(3) For our mission, a path to institutionalize & scale this new way of working, then put it in the hands of all of biopharma, all over the world - which, candidly, I find the most energizing.

Will AI x Bio cure diseases? Unequivocally yes, imo… but not by the route that gets the most airtime. Rather than timeline acceleration & compression, I see orchestration & parallelization as today's low-hanging fruit.

Most drug hunters see a handful of go/no-go calls in a career, & spend entirely too much of the rest trading emails (my inbox can attest).

Give them the bandwidth to run several programs at once, and the time to focus on scientific judgment, and I’m willing to bet they’ll make better calls. And at scale, better calls translate to more drugs, higher hit rates, broader patient impact, & likely, a disease or 10 or 100 cured. I couldn't be more excited to build toward that future @AnthropicAI.

Finally, a heartfelt thank you to BioTwitter/X. The end of anonymity is bittersweet, but if anyone’s earned a proper introduction, you all have.

It’s difficult to overstate how much this app has done for me: a decade of education, plenty of friends, many job opportunities, several mentors, a few investors, even a co-founder (fmr). It was the generosity of this community that inspired me to start posting 6 years ago, after just as many years lurking. I hope I’ve been able to pay it forward, in my own small way.

But, to be clear, Prof Oak isn’t going anywhere. Same beat, same themes, and now, a front-row seat to drug development at the frontier. Stay tuned for dispatches from the trenches; and remember...

Gotta Cure 'Em All

[youssef @ymkallel]

This is the standard of how agents will be deployed especially in regulated domains

I like to think of it as a high-fidelity “staging” env purpose-built for the real scenarios an agent is supposed to handle

On the standardization side, teams like @aiunderwriting are building certifications I expect to see greater adoption of and that can depend on such tests

[Berci Meskó, MD, PhD @Berci]

An FDA official just said that generative AI tools in medicine might get evaluated more like a physician, rather than a device or software.

He explained the difficulty in regulating gen AI brilliantly:

“We’ve been looking at software as a medical device for 20 years. We’re used to a paradigm where you have a piece of software — you can put in X, and it spits out Y — in a nutshell. ...Well, that’s pretty straightforward from an evidentiary standpoint.”

“We ended up putting out this idea that maybe these devices are starting to behave a little bit more like me as a physician than a heart valve or a defibrillator. I went through a system where, over time, I was evaluated on my knowledge and my clinical judgment and my communication ability and my ability to adhere to certain safety boundaries.”

Source: https://t.co/MIeA6wuRbz

[youssef @ymkallel]

@nlarusstone Agree, we’ve actually been seeing this firsthand and running it with customers we deploy with

Every agent <> workflow pair has a simulation environment and task/test set to validate it (across trajectories, not just the final output)

[youssef @ymkallel]

Personally been thinking abt this through the lens of regulated industries - they kind of worked to manage incentives between different parties of humans through a shared regulatory standard with verification mechanisms like independent audits

There could be interesting practical tests of alignment in these envs (eg in a simulated clinical trial) by designing certain scenarios where strong motives and pressure conflict with regulation or patient safety

[youssef @ymkallel]

@mehalKashyap @PanoptiveHealth Thanks mehal!

[youssef @ymkallel]

1/ We put 15 AI agents to the test in a simulated clinical trial and had them run oversight for a 4 month period

today we’re releasing ClinOpsBench: Trial Oversight from @PanoptiveHealth.

the first of many environments we’ve built based on our customer deployments for evaluating agents on real, long-horizon clinical development work.

[youssef @ymkallel]

@saurabh_shah2 Appreciate it saurabh 🙏 equally charismatic repost from a charismatic guy

[youssef @ymkallel]

7/ doing that takes a full stack of environments, verifiers and workflow data to evaluate, train and deploy reliable agents in this regulated domain

Check out our results, methodology writeup and sample trajectories for the public world here: https://t.co/SwfVS8kuz8

If you’re building or evaluating models for life sciences (or just interested in connecting) we’d love to chat!

[youssef @ymkallel]

6/ most of the time and cost in getting therapies to patients is still in clinical development. agents can take on a lot of that knowledge work, but only if we can rigorously measure and improve how they behave in real workflows

and importantly - understand where escalation for human hand-off is still important

[youssef @ymkallel]

ur in good hands @DoorDash

[youssef @ymkallel]

This is what real work ethic looks like

[youssef @ymkallel]

Nature is humbling

[Andrew Curran @AndrewCurran_]

This is a slide from Terence Tao's 'Math 2.0' public lecture yesterday evening at Caltech. I am highly confident that, if the cure actually worked, almost any Stage 4 cancer patient would tell you that human comprehension of its mechanism is completely irrelevant to them.

[youssef @ymkallel]

More candidates -> greater need for efficient and discriminating validation

The kpi becomes how early and cheaply we can identify and develop the ones worth investment

[Ryan Flinn @RS_Flinn]

I think we’ll see a new class of drug hunter emerge. AI tools, cheap super computers, large open source genomic data and remote robotic wet labs will make it possible for solo entrepreneurs to screen drug targets in silico then test top results in vitro and sell that to biopharma.

[youssef @ymkallel]

Of course there’s portfolio risk but one thing this may be discounting is that the cost per clinical “shot” can go down as well

AI across clinical development (and operations) can make trials cheaper and faster, and likely help kill unsuccessful programs earlier

Capital is always a bottleneck but a key metric should also be how much clinical evidence each dollar deployed generates

[youssef @ymkallel]

Clinical trials might be the most expensive and noisy long-horizon verifiers in science

Been working on this at @PanoptiveHealth alongside clinical and research teams

Excited to share more of what we’ve been building