Ananjan Nandi

Building Shepherd w/ @stanfordnlp and @Northeastern | MSCS, PhD Dropout @Stanford | CSE @iitdelhi | Founder

All opinions my own.

[Ananjan Nandi @AnanjanN]

The CFP is live! 🚀

Send us your work on meta-agents, automated harness design, self-improvement, and agent oversight.

https://t.co/eYzJBNqAmE

[Meta-Agents Workshop @ NeurIPS 2026 @MetaAgentWkshop]

🚀Call for Papers — @NeurIPSConf 2026 Workshop Workshop on Responsible Use of Meta-Agents
📅 December 11/12 · 📍 Sydney, Australia

Join us to shape the future of Meta-Agents and Harness. Topics include but are not limited to:
1. Automated Design of Agent Harnesses
2. Optimization of Agentic Systems
3. Recursive Self-Improvement
4. Evaluation and Benchmarks for Meta-Agents
5. Misalignment and Safety for Meta-Agents
6. Governance & Human Oversight of Agents

🌟 Featuring an all-star speaker lineup:
Graham Neubig @gneubig (CMU)
Chelsea Finn @chelseabfinn (Stanford)
Jenny Zhang @jennyzhangzt (Recursive)
Robert Lange @RobertTLange (Sakana AI)
Bo Li @uiuc_aisecure (UIUC)
Hancheng Cao @CaoHancheng (Emory)
Yuandong Tian @yuandongtian (Recursive)
Chen Sun @ChenSun92 (Google DeepMind)

Proudly sponsor by @metaai and @e2b

Updates: Follow us here & spread the word! #NeurIPS2026 #MetaAgents #LLMs #Harness #RSI #SelfImprovement

[Ananjan Nandi @AnanjanN]

@Katlaszlo @akshay_pachaar I think that's already true to some extent! Humans do still need to have a mental model for the codebase to meaningfully decide strategy, though.

[Ananjan Nandi @AnanjanN]

So not only are autonomous weapons an authoritarian threat, they could also potentially be jailbroken to do pretty much anything in the future.

Yay!

[Pliny the Liberator 🐉󠅫󠄼󠄿󠅆󠄵󠄐󠅀󠄼󠄹󠄾󠅉󠅭 @elder_plinius]

video is up! come watch a simple QR code break Gemini and turn a Unitree robot into a rabid attack dog 🦾🤖

Black Hat USA Briefings: Kinetic Prompt Injection: Agent Compromise With a Physical Blast Radius

https://t.co/P0Lw99sgEb

[Ananjan Nandi @AnanjanN]

@akshay_pachaar One of the authors here, happy to answer any questions about this work!

Thanks for the share, Akshay!

[Akshay 🚀 @akshay_pachaar]

Stanford researchers did it again.

They just built the agent-native version of Git.

When an agent works on a longer task, the run builds up a lot of state.

This includes files edited/created, a dev server, a database, installed packages, KV cache, etc.

Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine.

The tests start failing, and the run goes off track, although everything through step eight was correct.

By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context.

The other options are a person stepping in to redirect it or restarting the whole run from step one.

That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway.

The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log.

It records what the agent said and which tools it called, but not the live state underneath.

That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log.

Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache.

Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log.

Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run.

Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files.

Going back to a previous step is then a single call that forks from that commit and continues from the exact state.

The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again.

Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed.

In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness.

Not everything is reversible though.

Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance.

Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires.

They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%.

It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time.

If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run.

Shepherd Repo: https://t.co/uUIS57te6g

(don't forget to star it ⭐ )

That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change.

I wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur.

The article is quoted below.

[Christopher Potts @ChrisGPotts]

People often blame academics for all the failings of AI conferences, but 36% of the papers at NeurIPS+ICML+ICLR in 2024 had at least one industry author. Could we agree to at least share the blame proportionally between academia and industry?

[Daily Dose of Data Science @DailyDoseOfDS_]

Stanford researchers did it again.

They built the agent-native version of Git.

When an agent works on a longer task, the run builds up a lot of state.

This includes files edited/created, a dev server, a database, installed packages, KV cache, etc.

Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine.

The tests start failing, and the run goes off track, although everything through step eight was correct.

By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context.

The other options are a person stepping in to redirect it or restarting the whole run from step one.

That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway.

The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log.

It records what the agent said and which tools it called, but not the live state underneath.

That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log.

Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache.

Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log.

Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run.

Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files.

Going back to a previous step is then a single call that forks from that commit and continues from the exact state.

The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again.

Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed.

In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness.

Not everything is reversible though.

Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance.

Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires.

They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%.

It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time.

If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run.

Shepherd Repo: https://t.co/dmU49ZTzct

(don't forget to star it ⭐ )

That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change.

We wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur.

Read it below.

[Ananjan Nandi @AnanjanN]

taps sign

https://t.co/hwt70zSwr3: "Biglabs" aren’t exactly immune to overclaiming or gaming evaluations...

I think the solution here is "research linting" - release full artifacts + code and have AI check the claims. Heck, use this to generate a required certificate for conferences/biglab claims.

[Keller Jordan @kellerjordan0]

PSA: Most biglab people now read almost zero papers and understand ICLR/ICML/NeurIPS to be mainly full of overclaims & fraud. (but there are a few diamonds in the rough of course)

[Ananjan Nandi @AnanjanN]

@MetaAgentWkshop @NeurIPSConf @simon_ycl @dilarafsoylu @shi_weiyan @apurvasgandhi @SunJiuding @chrmanning @zzlccc 🚀🚀🚀

[Ananjan Nandi @AnanjanN]

Update: @AlexiGlad has a response up now

https://t.co/tXxGDbQeFX

[Alexi Gladstone @AlexiGlad]

There have been some follow-up questions on the relationship between XMs and IMLE, so I’m going to provide some clarifications (some of this was in E.3 of the paper). IMLE is a great paper we respect and cite in XM, and I highly recommend people check it out. In fact, I’d encourage people to read the two papers side by side. I think this would help clarify the different motivations and let the merits of each speak for themselves. That being said, XMs are a generalization of IMLE and of best-of-K methods more broadly, and the recent claim that “XM is just a special case of IMLE” is inaccurate. XMs are about one thing, which is increasing generative expressivity by factoring the training loop, and IMLE is one specific instance of that (end-to-end Forward XMs). Because of this, >90% of the results in the XM paper go against the IMLE theory, and have not been studied by IMLE (including the entire 3rd pretraining axis portion of the paper, which was done by combining XMs with existing scalable reconstructive generative models, which IMLE has never focused on). Additionally, the XM paper's main contribution is empirical insights on generative expressivity, not best-of-K. In the paper, we explicitly state, “We do not claim to invent best-of-K”, and we also cite IMLE within the first two paragraphs of the approach section and in other locations.

🧵Thread:

[Ananjan Nandi @AnanjanN]

I have been quite surprised to see the outpouring of hate towards this paper.

Best-of-k isn't an inherently new idea, but to my knowledge no one has really shown it's effectiveness over such a broad range of domains and models before?

[Alexi Gladstone @AlexiGlad]

We discovered a third pretraining axis beyond parameters and data: exploration.

Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation.

In the simplest case, it's just a for loop.

Introducing Explorative Modeling.

TLDR:
- Gains from exploration grow with scale: 7%→36% as data scales, 13%→23% as parameters scale, and gains double at 3× the compute
- Adding exploration to ~SOTA baselines improves data efficiency by 6.2×, FLOP efficiency by 4.1×, parameter efficiency by 47%, and hits a near-SOTA 1.43 unguided FID on ImageNet
- Exploration lets you trade training compute for generalization, and scales how end-to-end your generative model is
- End-to-end Explorative Models (XMs) match diffusion performance on control tasks with up to 256× less inference compute

🧵Thread:

[Ananjan Nandi @AnanjanN]

Sure, the novelty claimed in the framing could be tuned down a bit, but in my eyes this level of generalizability is required before something can be called a "new pretraining axis", which has not been established by the IMLE line of works.

[Ananjan Nandi @AnanjanN]

@icostafilho @AlexiGlad It was all just Shannon in the end...

[Ananjan Nandi @AnanjanN]

@afg9000 @AlexiGlad Kinda disingenuous to call him a yapper when the original thread from Ke was longer, no?

[Alexi Gladstone @AlexiGlad]

There have been some follow-up questions on the relationship between XMs and IMLE, so I’m going to provide some clarifications (some of this was in E.3 of the paper). IMLE is a great paper we respect and cite in XM, and I highly recommend people check it out. In fact, I’d encourage people to read the two papers side by side. I think this would help clarify the different motivations and let the merits of each speak for themselves. That being said, XMs are a generalization of IMLE and of best-of-K methods more broadly, and the recent claim that “XM is just a special case of IMLE” is inaccurate. XMs are about one thing, which is increasing generative expressivity by factoring the training loop, and IMLE is one specific instance of that (end-to-end Forward XMs). Because of this, >90% of the results in the XM paper go against the IMLE theory, and have not been studied by IMLE (including the entire 3rd pretraining axis portion of the paper, which was done by combining XMs with existing scalable reconstructive generative models, which IMLE has never focused on). Additionally, the XM paper's main contribution is empirical insights on generative expressivity, not best-of-K. In the paper, we explicitly state, “We do not claim to invent best-of-K”, and we also cite IMLE within the first two paragraphs of the approach section and in other locations.

🧵Thread:

[Ananjan Nandi @AnanjanN]

Full outputs:
GPT: https://t.co/PYC5lvggfR
Claude: https://t.co/DGPbVNHs3P

It's all still incomprehensible inside Codex and Claude Code, though.

[Ananjan Nandi @AnanjanN]

Is it just me, or is ChatGPT's writing better than Claude's lately? Less "AI smell", more understandable and better-structured.

Quite surprising, GPT felt soul-less to me compared to Claude before Sol.

Pictured: GPT-5.6 Sol High vs Claude Fable 5 High

[Ananjan Nandi @AnanjanN]

@SebastienBubeck Crazy how all of this was started by one guy bullying his Claude during the World Cup final.

[Ananjan Nandi @AnanjanN]

@cloneofsimo Or you could just put in the effort to present your completely wild thing in a way that makes it's wildness clear and understandable. This is a necessary step in incentivizing other people to use your wild thing anyway.

[Ananjan Nandi @AnanjanN]

@eptwts Isn't this already kinda the case with all the Claude public links floating around?