Bo Liu (Benjamin Liu)

9 days ago

AutoEnvScaling is a joint work with @BaolinPeng, @Benjamin_eecs, @Zhengyan_Shi, @zhaoywang_CS, Hao Cheng, @5000hui, @YaoWenlin, @natashajaques, @shi_weiyan and @JianfengGao0217 :)) @MSFTResearch × @Northeastern × @stanfordnlp × UW × UNC Paper: https://t.co/XZWuHVKCFo Github: https://t.co/pjxYN4YfA4 Page: https://t.co/grMw2jFRyH Built on slime (@slime_framework), miles (@radixark) and Harbor (@harborframework)

0

22

5

14

1K

Benjamin_eecs retweeted

elvis

@omarsar0

9 days ago

New impressive work from Microsoft and collaborators. And it's one of my favorite recent topics: RL environments. They show that Qwen3.6-35B-A3B goes from 41.1 to 53.3 on Terminal-Bench 2.1 by training on environments it designed itself. AutoEnvScaling gives environment design to a proposer agent that works in a terminal, looks at the rollouts of the model being trained, and builds new RL environments from them. The trained model's next rollouts guide the next round. Building environments in a terminal raises the share of valid environments up to 25 times over prompt-only generation, at up to 75% lower cost per valid environment. With GPT-5.6-sol or Claude Opus 5 as the proposer, Qwen3.5-35B-A3B scores 4.3 and 3.8 points above matched static environment pools. The gain shrinks when the proposer cannot see solver feedback. The pipeline also produces environments for software engineering, computer use, and GPU kernel work. https://t.co/o4d8tfErYD Chat with Paper: https://t.co/jC3MRnbAbk

omarsar0's tweet photo. New impressive work from Microsoft and collaborators.

And it's one of my favorite recent topics: RL environments.

They show that Qwen3.6-35B-A3B goes from 41.1 to 53.3 on Terminal-Bench 2.1 by training on environments it designed itself.

AutoEnvScaling gives environment design to a proposer agent that works in a terminal, looks at the rollouts of the model being trained, and builds new RL environments from them. The trained model's next rollouts guide the next round.

Building environments in a terminal raises the share of valid environments up to 25 times over prompt-only generation, at up to 75% lower cost per valid environment.

With GPT-5.6-sol or Claude Opus 5 as the proposer, Qwen3.5-35B-A3B scores 4.3 and 3.8 points above matched static environment pools. The gain shrinks when the proposer cannot see solver feedback.

The pipeline also produces environments for software engineering, computer use, and GPU kernel work.

https://t.co/o4d8tfErYD

Chat with Paper: https://t.co/jC3MRnbAbk

26

142

22

108

12K

9 days ago

@thewidechaos Good point, worth adding :))

0

42

9 days ago

a minimal RSI would be: a coding agent writes the repo that trains its successor, the pretrain, mid-train and post-train recipe plus a new harness. As a follow-up to SPADE, AutoEnvScaling automates one piece of that repo: post-training environments and curriculum for the terminal domain. The environment designer is now a coding agent: it reads the reasoning agent's rollouts, then builds, runs and fixes each environment in a sandbox. Building a terminal task and solving one are both coding, so one model plays both roles and gets better at both. Each checkpoint builds the tasks that train the next. Qwen3.6-35B-A3B goes from 41.1 to 53.3 on Terminal-Bench 2.1 :))

9 days ago

Recursive self-improvement needs agents that can build their own training environments. But researchers still part of the loop, making it hard to scale. We introduce AutoEnvScaling, which turns Data Flywheel into a terminal task: a proposer agent builds complete Harbor tasks and sandbox. It builds just like human researchers. This enables self-play RL for terminal agents: one model improves as both builder and solver, without human-in-the-loop.

simon_ycl's tweet photo. Recursive self-improvement needs agents that can build their own training environments. But researchers still part of the loop, making it hard to scale.

We introduce AutoEnvScaling, which turns Data Flywheel into a terminal task: a proposer agent builds complete Harbor tasks and sandbox. It builds just like human researchers. This enables self-play RL for terminal agents: one model improves as both builder and solver, without human-in-the-loop.

12

265

49

208

25K

4

65

7

33

3K

Benjamin_eecs retweeted

Natasha Jaques

@natashajaques

9 days ago

Our new paper pushes multi-agent recursive self-improvement even further. This time the proposer agent builds whole Harbor sandbox environments to challenge the solver agent. It learns from the solver’s successes and failures to update its own memories, skills, and tools, and build increasingly challenging---yet valid---tasks. We show powerful benefits both when using a proprietary model as the proposer, and in self-play, when both the solver and proposer are the same open-weight model. In this RSI setting, Qwen3.6-35B-A3B doubles its score on Terminal Bench 4, and performs so well on HiL-Bench it actually exceeds Claude Opus-5. More evidence of the power of multi-agent co-adaptation to unlock continuous improvement and emergent capabilities.

4

106

17

73

6K

9 days ago

@deeponailabs Each environment ships with its own verifier: tests plus a reference solution, both written by the Environment Designer. We use certain heuristics for QC: e.g. the reference must pass and an empty solution must fail

0

1

0

52

Benjamin_eecs retweeted

9 days ago

(2) Can AutoEnvScaling enable recursive self-improvement? We self-play on Qwen3.6-35B-A3B, since building an environment and solving one are both terminal tasks. Each new checkpoint builds the next round of training environments. Both roles get better: Terminal-Bench 2.1 rises from 41.1 to 53.3, while more of the environments it builds pass validation. The data flywheel closes: the model builds what it trains on and gets better at both building and solving.

simon_ycl's tweet photo. (2) Can AutoEnvScaling enable recursive self-improvement?

We self-play on Qwen3.6-35B-A3B, since building an environment and solving one are both terminal tasks. Each new checkpoint builds the next round of training environments. Both roles get better: Terminal-Bench 2.1 rises from 41.1 to 53.3, while more of the environments it builds pass validation. The data flywheel closes: the model builds what it trains on and gets better at both building and solving.

1

14

3

868

Benjamin_eecs retweeted

9 days ago

Recursive self-improvement needs agents that can build their own training environments. But researchers still part of the loop, making it hard to scale. We introduce AutoEnvScaling, which turns Data Flywheel into a terminal task: a proposer agent builds complete Harbor tasks and sandbox. It builds just like human researchers. This enables self-play RL for terminal agents: one model improves as both builder and solver, without human-in-the-loop.

12

265

49

208

25K

Benjamin_eecs retweeted

Anthropic

@AnthropicAI

10 days ago

New on our Engineering blog: Creating in-house source control. In May, we merged around 2,500 code changes per day. In September, we merged more than 30,000 per day. As a result, we became bottlenecked by developer infrastructure built for human (rather than agent) scale. Here, we share how we built a new version control system—and fully migrated our monorepo from git—in five months: https://t.co/wzNkwHs7IN

1K

26K

3K

20K

8M

Benjamin_eecs retweeted

Ao Qu

@ao_qu18465

12 days ago

Thanks for commenting! Reef is built to help agents continually improve through real-world usage. If anyone has live agent traffic and wants to collaborate, please let us know (even better if you’re interested in building benchmarks and datasets in this space)! One challenge we face is that current academic benchmarks aren’t sufficient for evaluating continual self-improvement through real-world usage lol.

1

41

3

10

2K

14 days ago

Yeah, love the versioning idea. We want agents to mine their own learning signal from their own experience, with the curriculum adapting as the world changes and reality providing the grounding and verification signal, guarded by the laws of physics and human values :))

0

1

0

21

16 days ago

Honored that our SPICE made it into this year's @stateofaireport :)) A self-improving system needs to ground its training signal in interaction with the external world to keep improving. Thanks @nathanbenaich!

Benjamin_eecs's tweet photo. Honored that our SPICE made it into this year's @stateofaireport :)) A self-improving system needs to ground its training signal in interaction with the external world to keep improving. Thanks @nathanbenaich! https://t.co/Btm2PYaVGZ

Nathan Benaich

@nathanbenaich

16 days ago

welcome to the 9th annual @stateofai report! ai building better ai, world models, physical ai, geopolitics, cyber defense and 12-month predictions. read it at stateof(dot)ai or watch my editor’s cut here. a few of this year’s biggest findings follow:

44

692

126

762

145K

6

45

8

9

3K

15 days ago

@nathanbenaich ♠️♠️♠️

0

2

0

30

15 days ago

@GChongkai @nathanbenaich 🫡

0

2

0

40