We're releasing Terminal-Bench-Science: a benchmark for evaluating AI agents on research workflows across scientific domains.
An ongoing Stanford-led community effort, built by the team behind Terminal-Bench together with scientific domain experts at research institutions worldwide. v0.1 has 70 tasks. Claude Opus 5 solves only ~30%.
1/n ๐
We built #DiscoveryAgent to keep scientists in control of AI-driven discovery. Each step is transparent and easy to audit and steer.
Experts working with DiscoveryAgent generated exciting findings across astronomy, economics and genetics!
Huge congrats to @oq_35, @james_y_zou & co. for the launch of DiscoveryAgent! ๐
Agents can increasingly handle clearly defined scientific tasks (as weโre measuring with Terminal-Bench-Science). That frees researchers to focus on the work that leads to real scientific breakthroughs: asking the right questions, making sense of the results, and steering where the research goes next. Thatโs exactly what DiscoveryAgent enables.
https://t.co/WQbKflZmF1
1/ AI agents can now accomplish many research tasks. But tackling frontier problems requires deciding which questions matter and when to trust results.
We're excited to introduce DiscoveryAgent to empower scientists in their pursuit of rigorous, open-ended discovery ๐งต
Introducing Gemini 4 Neon! Pushing the frontier across domains and punching above its model class in a meaningful way. To me, Neon feels like the right balance of intelligence, speed, and usability. Available now across @GoogleAIStudio, the API, and more!
Shout-out to Terminal Bench Science for placing #2! ๐ Itโs an amazing project led by @StevenDillmann that upholds top rigour, the highest standards, and realistic scientific tasks for frontier agents to benchmark on. congrats to the team on the ongoing recognition! ๐
We are currently building the next version of Terminal-Bench.
Creating a great task is very hard but worth the effort as Jason points out: a task sets a hill to climb which large amounts of resources are directed towards.
If you want models to get better at a capability that you care about, create a task.
We work 1-1 with authors from proposal to release. If you are at the top of your field or know someone who is and wants to participate in making Terminal-Bench, DM me!
A room full of AI researchers, developers, and engineers. The launch of Intelligence Index v5. A look at where AI evaluation goes next.
Highlights from our launch event in San Francisco.
@AnthropicAI reports an incredible score of 74.6% on Terminal-Bench-Science 0.1 for Claude Fable 5.2.
We're actively working on our next release - can't wait to test how this model does on our new, more challenging scientific research tasks!
Introducing Claude Fable 5.2 and Claude Mythos 5.2, designed to tackle the most daunting technical challenges.
Theyโre capable collaborators on ambitious projects in many fields, and show improvements in areas like coding, reasoning, and vision.
Terminal-Bench 4.1 is here with updated verifiers for all tasks.
In this release, we focused on:
- resisting reward hacking
- reducing false negatives
- improving reproducibility
Re-grading the 4.0 leaderboard trials with the 4.1 verifiers, resulted in ~1% of rewards flipping.
Leaderboard scores remain within previously reported confidence intervals.
Congrats to @bingran_bry & co. on FrontierPhysics ๐โ๏ธ
Another strong AI-for-science eval, this time for end-to-end physics research. Great to see more benchmarks built on real research workflows.
Introducing FrontierPhysics - a benchmark evaluating agents for end-to-end physics research
Our core team from physics labs at Berkeley, Caltech, and Harvard recruited 42 physicists from 26 institutions. 28 of them published papers on Nature, Nature Physics, PRX, PRA*
1/7 ๐งต