Justin

Locus @Intology

[Justin @chojustin]

Locus sets SOTA on automated post-training capabilities and produces a model that's in production facing millions of users, completely end-to-end!

Here's a glimpse to the endless possibilities we're materializing with our research on automated scientific discovery:

[Intology @intology]

The models are improving the models.

Locus, our automated AI research system, is SOTA on PostTrainBench and post-trains Qwen3 base models that surpass the human post-trained Qwen3 model.

Today, LLMs post-trained end-to-end by Locus are in production to millions. 🧵👇

PostTrainBench evaluates agents' ability to post-train models on various domains given 10 H100 hours.

We extend PostTrainBench via PostTrainBench+, which has a greatly expanded compute budget that provides clearer signal on automated post-training capabilities. We find that thousands of H100 hours help distinguish methods' performance post-training Qwen3 1.7B-Base models, and that Locus scales best. In this setting, modes trained by Locus collectively surpass the perforamce of the offical human post-trained Qwen3 1.7B model.

In a test of generalization, we ran Locus on all live Kaggle competitions with prize money and public leaderboards. After 16 days, Locus achieved the 4th highest average rank among all participants.

[Avijit Thawani (Avi) @thawani_avijit]

by far the most important result on AI for research this year!

[Shriram @shriramc1]

I recently joined @intology because it’s clear that autonomous discovery systems can now accelerate progress on hard research problems, and I’m excited to scale Locus to solve more complex tasks, both in the sciences and beyond!

[Rahil Parikh @ParikhRahil]

After 4+ years building models at @AmazonScience (Alexa + AWS AI), I joined @intology a few months ago!

We’re automating AI R&D. Here are our results showing Locus building scientific AI models that beat SOTA expert solutions in molecular design

[Justin @chojustin]

We went from SOTA on autonomous post-training to frontier performance on autonomous molecular design (eg antibody design) in just a couple of months!

What's exciting about these results is that Locus achieved them via improvements in domain-agnostic auto-research capabilities:
- Diverse exploration
- Better Generalization to avoid overfitting
- Resilience to noise
- High sample-efficiency to learn from data-scarce tasks

This marks a crucial milestone towards building auto-research systems that can closing the loop for scientific challenges that have even larger search spaces and longer, more expensive, and noisier evaluation cycles.

[Intology @intology]

Locus sets the frontier for automating molecular design

Our automated research system Locus develops scientific AI models that outperform SOTA expert solutions

Unlocking the potential of scientific AI requires leveraging proprietary data. Locus achieves this autonomously 🧵👇

--

We test Locus across a number of tasks related to molecular scoring and design: training molecular property prediction models for a proprietary task, fine-tuning Boltz-2 to predict small-molecule potency, and building antibody design workflows around BoltzGen, Boltz-2, ESMC, and ESMFold2.

Each of these settings are real-world AI for science use cases, where tasks are often out-of-distribution for existing foundation models and limited in data, while featuring expensive and noisy evaluators. Compared to state-of-the-art baselines and LLM agents set to optimize these tasks, Locus achieves higher performance. Additionally, Locus generalizes better despite limited data, finds more sample efficient solutions, and explores a higher diversity of solutions.

[Alex Spangher @AlexanderSpangh]

🚨New work: A SOTA prompt optimization framework + a novel unsupervised measure of prompt performance.

Written rubrics and constitutions are crucially important for AI task improvement, safety following, pluralism and adaptation.

But some of the things we might want a rubric to do are difficult to put into words; how would you even describe “comic voice”? I’m human, albeit not that funny, and I'm totally not sure what that means.

What kinds of written constructs can you expect your judge to do well on and how good can you expect it to be?

[Alex Spangher @AlexanderSpangh]

So excited for this research! I've always been annoyed at the supremacy of verified rewards, and now we can finally show it.

In this work, we develop ways of measuring: how much of a reward signal is verifiable, how much can we put into words, and how much is shared but tacit, or impossible to articulate?

[Justin @chojustin]

@jaspreetranjit_ @EricRicePhD @swabhz woohoo congrats!!

[Justin @chojustin]

@aryaman2020 did not expect twosome to be part of this list.

congrats Aryaman!

[Justin @chojustin]

@jackson_stokes Lack of good evals
Lack of good training data

[Justin @chojustin]

@poteto how have you scaled up your ambitions

[Justin @chojustin]

@JayKapoor I would happily settle for second. his grandparents demoted me to fourth tier

[Justin @chojustin]

@AIMind_Ai prompt or it didn't happen

[Dhananjay Ashok @Dhananjay99Ash]

Introducing GameBoyWorlds, a new challenge testbed for self-improvement in embodied video games:

Paper: https://t.co/05Fpom1OM1
Website: https://t.co/qFUZfld80b

[Intology @intology]

Thank you @jackclarkSF for covering our work with Locus!

"AI startup Intology, whose goal “is to automate R&D”, has released a new version of Locus, software it has developed to turn LLMs into capable researchers. The new version of Locus is able to get a score of 44.7% on PostTrainBench, a benchmark which sees how well AI systems can take an open weight model and improve its performance above its baseline."

"Posts like this highlight how we are under-eliciting today’s AI systems for their ability to automate AI R&D - especially striking is how the company can jump the performance of Opus 5 by 10 absolute percentage points"

[Ron Arel @ronusedh]

Thanks to @jackclarkSF for covering @intology's work on Locus!

"Posts like this highlight how we are under-eliciting today’s AI systems for their ability to automate AI R&D - especially striking is how the company can jump the performance of Opus 5 by 10 absolute percentage points"

Full blog: https://t.co/JeKSHTB1xP

[Justin @chojustin]

@ParikPatelCFA superalignment

[Intology @intology]

Thank you to @tbpn for having us on last week. Ron Arel (@ronusedh, co-founder of @intology) spoke on our research direction — our past work, and our new state-of-the-art results on automating post-training.

[Hardik Bhatnagar @hrdkbhatnagar]

Super cool results on PostTrainBench!

They also introduce a very interesting setting of PTB where the agent hours are extended by a large margin (4500 GPU hrs!)

[Justin @chojustin]

@j_dekoninck @paradite_ https://t.co/KETURZMew7

fyi qwen also recommmends 0 for math