Gamow Labs

Democratizing world-class diagnostics for all babies

[Gamow Labs @GamowLabs]

@GoogleDeepMind gave us early access to Gemini 4. which achieved first place on our clinical genetics benchmark RareBench and scientific experiment design LabBench.

Thank you to Google for the early access and congrats on the launch!

[Daniel McKinnon @danielmckinn0n]

Big G is back! 🙇

@GoogleDeepMind Gemini 4 takes the 👑 on @GamowLabs RareBench and LabBench by a healthy margin over previous champ Opus 5.5.

A tip of the hat to @melvinjohnsonp, @taotu831, @divy93t, and the rest of the science team for this very strong release.

We've moved essentially all of our biology and genomics research work to Gemini 4 in agy with science skills since joining the EAP program and look forward to seeing if this model can help diagnose some new kids!

[Pushmeet Kohli @pushmeet]

In rare disease research, scientists at @GamowLabs used Argon to analyse de-identified genomic data samples from newborns with rare developmental lung disorders. It successfully pinpointed a chromosomal rearrangement that routine software missed for years, and explained why it went undetected. Read more here: https://t.co/Mga7ct8s5f

[Daniel McKinnon @danielmckinn0n]

Muse 🍉 Wonder @GamowLabs RareBench results are hot off the press. They are... disappointing.

We observe a small regression (empirically not stat stig, but we don't run replicates) as compared to Muse Spark 1.3 Max and performance far behind the Astra/Opus frontier.

It's possible that @AIatMeta is simply optimizing for different tasks, but RareBench generalizes surprisingly well and imo yields a better list of top models than more easily gameable public indices.

It is a verifiable, long-time horizon, and challenging eval that requires dozens of diverse tool calls in a domain that I'm confident no one is overfitting to yet.

Note that this is the weaker checkpoint released today. We will benchmark the stronger model, Muse Wonder Full (lol great name), as soon as API access is granted.

[Daniel McKinnon @danielmckinn0n]

I had a great in-depth conversation about @GamowLabs last week with @8teAPi and @labenz on @AI_in_the_AM that included an impromptu lab tour.

Full episode is live if you're looking for something to listen to on your way to work tomorrow.

https://t.co/KV1L1HpBbC

[Daniel McKinnon @danielmckinn0n]

We don't have the eval you've all been waiting for yet (hurry up and approve our early access request, @GoogleDeepMind!), but both @OpenAI and @AnthropicAI cooked with their recent cheaper releases.

On @GamowLabs RareBench, GPT-6.1 Sol nearly matched Astra at a quarter of the price, placing it solidly on the cost Pareto frontier.

Sonnet 5.5 also pushed the frontier, significantly outperforming Astra as a fraction of the cost of Opus.

The level of intelligence we can buy with each dollar is becoming truly astounding.

[Daniel McKinnon @danielmckinn0n]

Really fun conversation with @labenz and @8teAPi this morning on @AI_in_the_AM where we got to spend a bit more time discussing the technical details of @GamowLabs approach and paid a quick visit to our hard-working robot.

[Gamow Labs @GamowLabs]

Watch @danielmckinn0n, live now! 🗣️🎙️💻🧬🧠🚀🚀🚀

[Nathan Labenz @labenz]

Up now on AI:AM: Daniel McKinnon (@danielmckinn0n), CEO of @GamowLabs.

Their agent reads a sick newborn's whole genome to find the variant behind it. They also run RareBench, a scoreboard of which frontier model is best at clinical genetics.

https://t.co/2BCCZ0NV0u

[Daniel McKinnon @danielmckinn0n]

Looking forward to chatting AI and genomic medicine at @GamowLabs with @labenz and @8teAPi on @AI_in_the_AM at 10:15.

If we are lucky, our new robot will make a guest appearance.

[Gamow Labs @GamowLabs]

Benchmarking the frontier shows that Astra excels in both raw score, and in wall-clock time per task.

We are working to help the frontier advance along these axes, which we believe is critical to accelerating biological discovery.

Please reach out to work with us and learn more.

[Gamow Labs @GamowLabs]

Can AI agents decide what experiment to run next?

We present LabBench, a benchmark built to challenge agents to choose and design experiments for wet-lab biology.

Our core finding: agents struggle to commit to the next most-important experiment, despite considerable domain knowledge and expertise demonstrated in other parts of the eval.

[Gamow Labs @GamowLabs]

We organize our benchmark around core skills such as quantitative reading, evidence integration, and experimental design.

When presented with the task of replicating the next experiment that a human expert recommended, agents fall short disproportionately.

[Daniel McKinnon @danielmckinn0n]

We have a new 👑 of @GamowLabs RareBench. @AnthropicAI Opus 5.5 smashes the competition with 52%, adding 6 pp over the previous champ, Fable 5.1.

Part of this is because we report Fable 5.1 inclusive of safety-driven Opus downgrades. Now Anthropic has approved us for the Life Sciences Verification Program, we will be able to generate a pure Fable 5.1 score.

Other changes of note include confirmation of a somewhat widely reported @spacexai @grok regression with 4.7 losing 5 pp to 4.6.

We also interestingly observed an @OpenAI regression with GPT-6 Sol slightly underperforming GPT-5.6. This is likely within the error of this eval (those measurements coming soon!), but I did expect the Astra performance to translate into the latest Sol models.

Luna surprisingly is at the back of the pack, which probably indicates that there is some minimum scale required for this task.

[Daniel McKinnon @danielmckinn0n]

My first LLM project was in 2022 at Meta where I needed to figure out if we could use OPT-175B as a first-line classifier to flag newly problematic content ahead of training a supervised model.

Seeing Jev attract this hype felt like a full-circle moment. Isn't this just LLM + constrained decoding?

I haven't seen many benchmarks, so we compared Jev to GPT-nano for a large-scale document classification project.

Jev and GPT-nano scored identically (coincidence?!?!) while Jev was 5x faster and surprisingly 20% more expensive.

For us, the speed tradeoff is worth it, but if we needed to run this task continually, I suspect that we could get better/faster/cheaper with a fine-tuned Qwen or Gemma model. I'm really not sure what the excitement is all about.

Can someone ELI5?

[Daniel McKinnon @danielmckinn0n]

We learned that lesson fast at @GamowLabs! More soon.

[Daniel McKinnon @danielmckinn0n]

If you're more of a podcast person and interested in learning more about what we're up to at @GamowLabs, my @MTSlive episode just dropped on their feed.

@sophiadew is a great host and I think really helped me tell my story in a way I am proud of.

https://t.co/1x4uqo24or

[MTS @MTSlive]

Gamow Labs founder @danielmckinn0n reveals how o3 diagnosed the rare genetic disease that killed his son after a leading neonatal genome lab missed it:

"Our origin story is pretty sad, but unfortunately, it's not unique. Five years ago, I lost my first son to a rare genetic disease. When we say rare disease, we're not talking about rare disease in aggregate. When you add up all of these, we're losing a tremendous number of children to rare diseases that could be diagnosed if our tools were better."

"Two weeks in the NICU we realized something was seriously wrong. The staff ordered a whole genome test, gold standard diagnostic. It came back negative."

"At eight weeks old, we made the impossible decision to withdraw care. Months of diagnostic odyssey, we found the diagnosis for his condition. We also found out it was inherited, which impacted our ability to start a family."

"We tragically had a second loss very late for genetic reasons. My now healthy one-year-old son, Warren, was 16 weeks in utero. Another whole genome sequence came back also negative."

"My whole background is in AI. I called all the labs and requested all the raw data under my HIPAA rights. I wrote what we'd call an agentic loop, pre-Codex, pre-Claude Code. I put o3 in a harness and said, figure this out."

"My jaw hit the ground when I saw that it diagnosed my first son, Owen, that the best neonatal whole genome lab missed a few years ago. At that point I knew I wanted to dedicate my life to solving this problem."
@GamowLabs @OpenAI

[MTS @MTSlive]

Diagnosing a rare genetic disease is a better AI problem than writing.

Gamow Labs founder @danielmckinn0n on why genome interpretation is verifiable the way math and code are:

"It's not like these people have abacuses, but the reads come in and they'll use various different pieces of software to do various different parts of that pipeline."

"At the end, they come up with a packet, and this packet might contain 100, it might contain 10, it might contain one, it might contain zero different variants."

"It was the classic human-driven deep research project, but it's verifiable."

"AI models suck at writing 'cause no one knows what good writing is. This is definitely causative, and here's a very structured line of reasoning to get there."

"It's a perfect task for AI, and the models are getting quite good at it, and I think how we leverage the models is the best in the world."
@GamowLabs

[Daniel McKinnon @danielmckinn0n]

Looking forward to joining @sophiadew in a few hours to talk about @GamowLabs is working at the intersection of AI and genetics to bring the best healthcare to all babies and help eliminate rare disease.

Please tune in if you’re interested!

[Gamow Labs @GamowLabs]

🚨🚨🚨 @danielmckinn0n will be talking about the future of AI and genetics at 11:30am ET on @MTSlive. Everyone will be talking about this
🗣️🎙️💻🧬🧠🚀🚀����

[Daniel McKinnon @danielmckinn0n]

Looking forward to joining @sophiadew in a few hours to talk about @GamowLabs is working at the intersection of AI and genetics to bring the best healthcare to all babies and help eliminate rare disease.

Please tune in if you’re interested!