Closed-loop discovery is starting to look real.
This is autonomously optimizing models for molecular property prediction, potency, and antibody design on proprietary/limited data and beating generalist agents (Codex, Claude Code) on generalization and sample efficiency.
So cool
Last summer, I joined @intology, leading the effort to connect autonomous AI research to scientific discovery.
Today, we’re sharing work on autonomously adapting frontier scientific models to proprietary datasets and our broader vision for what's next.
🧵(1/6)
We went from SOTA on autonomous post-training to frontier performance on autonomous molecular design (eg antibody design) in just a couple of months!
What's exciting about these results is that Locus achieved them via improvements in domain-agnostic auto-research capabilities:
- Diverse exploration
- Better Generalization to avoid overfitting
- Resilience to noise
- High sample-efficiency to learn from data-scarce tasks
This marks a crucial milestone towards building auto-research systems that can closing the loop for scientific challenges that have even larger search spaces and longer, more expensive, and noisier evaluation cycles.
Locus sets the frontier for automating molecular design
Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions
Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇
--
We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.
Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.
In our most recent results, we demonstrate one of the first cases of generalizing from autonomous AI research (ie post-training) to auto-research for antibody design and potency prediction.
Excited to see where we can go from here!
Excited to share new results from Locus on molecular design AI R&D problems!
"Scientific ML" problems have been very useful for our AI R&D research program in overcoming evaluation variance.
Locus is now verified on the official PostTrainBench leaderboard!
Thank you to @hrdkbhatnagar, @full__rank, and @maksym_andr for reviewing and uploading our solutions on the 10-hour setting.
Thank you to @tbpn for having us on last week. Ron Arel (@ronusedh, co-founder of @intology) spoke on our research direction — our past work, and our new state-of-the-art results on automating post-training.
strong results on PostTrainBench from @intology! our team verified them, so the improvements are real.
however, i'm still not very bullish on harness engineering. i think it's a transient source of improvements like prompt engineering used to be 2-3 years ago. i.e., yes, you can get a bit of a boost (and it might matter in practice for some applications!), but it will likely not matter medium/long-term (~1-2 year horizon). a general-purpose LLM + a general-purpose harness (basically, better versions of Claude Code / Codex) is most probably all you need.
The models are improving the models.
Locus, our automated AI research system, is SOTA on PostTrainBench and post-trains Qwen3 base models that surpass the human post-trained Qwen3 model.
Today, LLMs post-trained end-to-end by Locus are in production to millions. 🧵👇
PostTrainBench evaluates agents' ability to post-train models on various domains given 10 H100 hours.
We extend PostTrainBench via PostTrainBench+, which has a greatly expanded compute budget that provides clearer signal on automated post-training capabilities. We find that thousands of H100 hours help distinguish methods' performance post-training Qwen3 1.7B-Base models, and that Locus scales best. In this setting, modes trained by Locus collectively surpass the perforamce of the offical human post-trained Qwen3 1.7B model.
In a test of generalization, we ran Locus on all live Kaggle competitions with prize money and public leaderboards. After 16 days, Locus achieved the 4th highest average rank among all participants.
Coding agents struggle to do AI research.
Our analysis on the NanoGPT speedrun show they rarely attempt ambitious ideas, instead opting for hyperparameter tweaks. Releasing our framework as a benchmark here!
Can coding agents do research?
We release NanoGPT-Bench, an internal eval we’ve used to test agents on an AI R&D problem with months of human progress
Codex, Claude Code, Autoresearch recover only 9.3% of human progress, mostly tuning hyperparams & ignoring algorithmic research
NanoGPT-Bench is built on the NanoGPT Speedrun, a popular LLM pretraining competition to minimize the training time of a GPT-2 style model. Existing human submissions constitute nearly 2 years of work. To control for dependencies and contamination in frontier models, we standardize evaluation to a 5-month window of world records. Evaluation is fully autonomous and end-to-end, with no human intervention or internet access. 🧵
New NanoGPT Speedrun WR at 105.9s (-1.0s) from @soren_dunn_ , with a triton kernel to fuse the logit softcap and multi-token prediction cross entropy calc. Interestingly, Soren mentioned that their autonomous system Locus at Intology discovered and implemented the improvement. https://t.co/eU5UZT3nYJ