Weβre continuously adding new models to the BioResearchBench leaderboard!
Results for more cost effective models, including Gemini 4 Hydrogen and Muse Ember, are now available!
In the last week, @GoogleDeepMind and @Meta each released workhorse models that have set the cost-performance frontier for long-horizon biology work.
Hydrogen's performance is especially surprising and great to see! It was very close to Opus 5.5 on most tasks but (unsurprisingly) had some challenges handling the very long-horizon tasks leading to time-bound failures.
Read more: https://t.co/RAye2J4cXh
Fable 5.2 is the new SOTA model on BioResearchBench, showing a 5 p.p. improvement over Opus 5.5., including solving 5 tasks previously not solved by any model. Congrats to
@AnthropicAI@amirshanehsaz@mgdurrant@nc_frey@JCoolScience and team
Introducing BioResearchBench, a new benchmark focused on scientific reasoning in biology from @OfficeHours built by 100+ PhD biologists in our network.
For models and agents to materially speed up scientific discovery, scientists need to be able hand off work that would take them hours to complete, come back, and find it done right without mistakes.
This work shows that frontier models arenβt there yet. They still struggle to stitch multiple steps together, get the computations right, interpret the data correctly, handle uncertainty, and draw the right conclusions.
We built 50 long-horizon research tasks across 10 subdomains, from genomics and cancer biology to neuroscience and immunology. Each uses multiple data files, takes an expert hours to several days to complete, and is independently reviewed by at least 2 PhD biologists for verifiability and scientific accuracy.
Every frontier model passes fewer than 1 in 5 tasks:
Claude Opus 5.5: 18%
Gemini 4 Argon: 13%
Muse Wonder: 11%
GPT-6 Astra*: 4%
Grok 4.7: 4%
The top models met ~80% of individual rubric criteria, but completed the full end-to-end workflow correctly <20% of the time. Half of the tasks were not solved by any model.
The biggest gaps point to where models need to improve next: computation and data handling (failures on 67% of tasks), handling uncertainty (55%), and reporting complete answers (55%).
An important note on scoring: GPT-6 Astraβs score is skewed lower because every task had to stump Astra (SOTA at the time) during development before it made the final set.
Paper, leaderboard, and per-task results:
https://t.co/RAye2J4cXh
@samar_abedrabbo@alexpryhoda