Would love to hear opinions: is one-step fidelity the right target for an RL simulator or does "safe to train" need more?
Paper: https://t.co/rken5tYPF5
Code: https://t.co/Ims982R77A
Benchmark:https://t.co/dNdyUJuVcs
RL increasingly depends on environments agents act in and are rewarded by. Building fresh environments for every new domain is slow. An alternative to this is to let a language model be the environment. The agent sends an action and the model predicts what real software would have returned.
Today's models are nowhere near good enough to simulate the environments we train RL agents in. GlobeBench shows where they break. Which state transitions models can and can't reproduce.
w/ @nick_saban20, @getdarshan & @mariyaivasileva
Today we're open-sourcing GlobeBench, 1,632 tasks that test whether LLMs can simulate RL environments well enough to train agents on.
The top model, GPT-6 Astra, gets 49.6% right. No other model of the 20 we tested reaches 30%. 🧵
3) Thinking longer doesn’t improve the simulation, it just elaborates the wrong one.
You’d expect the model that “thinks” more to do better. The opposite holds. On the harder items GPT-6 Astra spends a median of 8.3k tokens and tracks the true state, Grok 4.6 spends 39.9k nearly 5× more and still carries the wrong capability, producing a longer chain of internally-consistent but wrong steps. The separating factor is fidelity. Astra reports the exact hidden value on 84% of requests, the others on 2–8%.
Today we're open-sourcing GlobeBench, 1,632 tasks that test whether LLMs can simulate RL environments well enough to train agents on.
The top model, GPT-6 Astra, gets 49.6% right. No other model of the 20 we tested reaches 30%. 🧵