Can reusable research skills help AI generate better biomedical research plans?
Our exploratory study compares native LLMs and skill-augmented agents across six frontier models on an NSCLC transcriptomic biomarker task. https://t.co/xYGs2l9iig
Everyone is asking how to build better AI research agents.
Our latest study may suggest a different question:
Preprint: https://t.co/qk5hbXvO03
How do we evaluate them?
In a blinded human evaluation of medical research agents, Skill-Augmented Agents showed positive signals in research quality.
But here's the surprising part:
The disagreement between expert reviewers (0.67 points on average) was larger than the observed improvement from skills (+0.39).
Even more striking, expert inter-rater reliability was negative (ICC = -0.15).
The bottleneck may no longer be agent capability.
It may be the evaluation itself.
As AI agents become increasingly capable, building reliable evaluation frameworks could become one of the most important challenges in AI for Science.
#AIAgents #AIforScience #Evaluation
To maximize the potential of AI Scientist, our team found these matter as we are building open science
1. Give it narrow, well-defined problems instead of vague topics. The more scoped the task, the better it tends to perform.
2. Lean hard on the Claim Verifier and audits, treat them as free, fast peer review and fix issues early.
3. Keep a human in the loop for high-level direction and novelty judgment, because the system is strong at execution and traceability but still needs guidance on what’s actually worth pursuing.
4. Use the parallel exploration to test multiple directions quickly, then focus your own effort on the most promising branches.
Experience AI Scientist, a novel multi-agent system designed to automate the end-to-end scientific research pipeline. Meet Jinsung Yoon and Rui Meng today at 6:00pm at the Google booth (#B206) to learn how complex workflows reliably accelerate AI discovery. #ICML2026
6 months of Claude Max 20x, on us.
We're expanding Claude for Open Source to more of the community.
If you're a maintainer, a core contributor, someone landing PRs across the ecosystem, or someone keeping a critical package alive, apply today!
Anthropic: "We found a neural pattern in Claude. It's interesting but we don't know what it means."
Tech Twitter: "AI IS NOW CONSCIOUS."
Anthropic: "...we don't know if it's consciousness."
Substack: "I asked Claude if it dreams. It said yes. Here's what that means for you, your job, and the meaning of life."
New Anthropic research: A global workspace in language models.
Of everything happening in your brain right now, only a tiny fraction is consciously accessible—thoughts you can describe, hold in mind, and reason with.
We found a strikingly similar divide inside Claude.
For wet lab researchers though, if this workflow really works at scale, it could reduce a lot of the trial-and-error before the experiment even starts.
The only downside is the cost. For smaller labs or individual researchers, the pricing could become a real barrier to adoption.
🚨 Anthropic Releases Claude Science workbench:
• Specialised AI workbench tailored completely for scientific research
• Natively renders and interacts with complex 3D protein structures
• Dynamically manages and scales compute pipelines on remote HPC clusters
• Designed to accelerate breakthroughs in biology, chemistry, and advanced physics
Things like this are exactly why AI is so interesting and crucial.
what do you think these biology/science models will make possible in the next 10 years?
I think it depends on where you think wet lab work begins.
Choosing the right protocol, checking whether someone has already tried a similar experiment, comparing conflicting papers, selecting controls, planning follow-up experiments, interpreting unexpected results, and troubleshooting failed experiments can easily take far more time than running the experiment itself.
Claude Science helps reduce the time spent around the wet lab.
AI is transforming drug discovery, but until now there hasn't been an independent benchmark for the computational tasks behind early-stage research.
Together with @phylo_bio, we built DrugDiscoveryBench to evaluate how today's leading AI models perform on the core tasks researchers rely on and where they still fall short.
Even the best-performing models completed just over half of the benchmark's tasks.
Reproducing a single paper already takes 100M+ tokens.
That makes reasoning efficiency worth paying attention to.
It also puts more emphasis on workflow design, caching, checkpointing, and context management, where a lot of those tokens can be saved.
We asked Claude Science to reproduce a high-impact biomedical paper.
11 hours. 2 skills. ~105M tokens.
A few observations from the process:
• Before writing a single line of code, Claude read the paper and asked clarifying questions: How closely should I reproduce it? What outputs do you expect? Which parts are realistically reproducible?
• It spent nearly 3 hours just setting up the environment (Conda, R packages, dependencies). Honestly, that's probably the most human part of the whole process. One package version mismatch or dependency conflict can easily cost hours.
• Data collection wasn't seamless. GEO and TCGA were available. GeneCards wasn't. MSigDB and MitoCarta were only partially available, so some inputs differed from the original study.
• Differential expression analysis didn't initially match the paper. Using the published thresholds produced very different results. Claude inferred that additional filtering steps were likely omitted from the manuscript, adjusted the thresholds, and ultimately identified 15 significant genes with the same direction of effect.
• Not everything could be reproduced exactly. Consensus clustering still differed, and pRRophetic was replaced with oncoPredict, its newer counterpart from the same research group.
• The workflow finished with all visualizations plus a report that explicitly documented every discrepancy and limitation instead of pretending they didn't exist.
Final usage:
Opus: 555 calls / 91.5M tokens
Sonnet: 273 calls / 13.1M tokens
We'll also be running side-by-side comparisons across different models once our project is ready.
@BoboHxx tested Claude Science by reproducing the full computational workflow of a breast cancer bioinformatics paper.
It handled the long, multi-step pipeline quite well, but also transparently flagged the parts that couldn’t be exactly replicated, such as upstream gene sources and threshold choices.
The discussion around auditability and provenance for scientific AI agents is especially useful. Worth reading.
#ClaudeScience #AI #Bioinformatics
@BoboHxx tested Claude Science by reproducing the full computational workflow of a breast cancer bioinformatics paper.
It handled the long, multi-step pipeline quite well, but also transparently flagged the parts that couldn’t be exactly replicated, such as upstream gene sources and threshold choices.
The discussion around auditability and provenance for scientific AI agents is especially useful. Worth reading.
#ClaudeScience #AI #Bioinformatics
Hi there,
We’re building Integrated Research Environment (IRE) for Bioinformatics data analysis and clinical data analysis
Our open-source work has crossed 1K stars, and we’re preparing a more visible launch around reproducible, reviewable analysis workflows.
We think your audience could be a strong fit: people who care about tools that move beyond demos into real workflows.
Could you share your current rate card and available collaboration formats?
Scientific discovery shouldn't be tied to a single closed platform.
Open Science is building an open, model agnostic workbench where workflows are inspectable, reproducible, and portable.
Launching soon!
#claude#claudescience
To make scientific workflows inspectable.
I’m building something for researchers who need more than a chat box.
Trying something: an open-source AI workbench for scientific discovery.
Not a polished product yet. Just the foundation: Electron + Vite + React, ACP agent protocol, shadcn/Radix UI, and a local app starting up.
Open Science is an open-source, model-agnostic AI workbench for scientific discovery: chat + artifacts + connectors + compute.
Still early. Today: package.json, ACP, Electron/Vite, local app booting.
The recent launch of Claude for Science is another reminder that AI is becoming an essential part of scientific research. #claudescience
We see that as an exciting step but we also think there's another conversation worth having. Helping researchers work faster is important. Helping research itself become reusable may matter even more. #openscience
Scientific progress has always depended on shared methods, reproducible workflows, and knowledge that compounds across generations of researchers. AI shouldn't stop at answering questions inside a chat window. It should help turn research workflows into shared infrastructure that anyone can inspect, improve, and build upon.
That's why we're building Open Science.
https://t.co/poXdpI9Vs5
Hard guardrails didn’t just nerf Fable 5
They proved why we can’t keep trusting closed systems for actual research.
Open science has to win here.
#claudescience#openscience#fable5
We’re all becoming vibe coders of science thanks to claude science.
BUT whether the gate for science is open or locked behind platforms still matters. 😅
Less closed science plz tho
https://t.co/T2IX9U4QnO
#openscience#claudescience
you don't understand how dangerous garage biotech is about to become
Anthropic casually launched Claude for Science
A research workspace with:
- 60+ scientific databases
- artifact tracing to source papers
- on-demand computational environments
People were already doing ridiculous things before this.
- designing drug candidates from home
- sequencing genomes on kitchen tables
- building personalized cancer vaccines
- running liquid-handling robots with Claude Code
Every new AI capability compounds on top of the last one.
The minimum viable biotech startup just keeps getting smaller.
bio/acc
Claude Science lowers the barrier to scientific research, which is genuinely valuable.
But lowering the barrier shouldn't mean moving more of the scientific workflow into a single managed ecosystem.
Science needs to become more accessible without becoming more dependent.
If you prefer more accessible direction for science👇
https://t.co/pWvi5gFGZw