Vmax

RL research focused on open-ended learning

[Vmax @VmaxAI]

1/ 🎮 Can training on games improve LLM reasoning?

Introducing GlyphBench: 360+ game tasks for language-model RL.

We use it as a benchmark, a research tool, and a post-training suite.

🧵⬇️

[Vmax @VmaxAI]

@daveey @creus_roger @Cote_Marc @GlenBerseth @pcastr 🫡

[Vmax @VmaxAI]

11/ Led by @creus_roger, in collaboration with @Cote_Marc, @GlenBerseth and @pcastr at @Mila_Quebec & @UMontreal.

📄 Paper: https://t.co/fdhCxLLAfF
💻 Code: https://t.co/Yw6ARxr7tu
🌐 Blog: https://t.co/P7cehJD1Xr

[Vmax @VmaxAI]

1/ 🎮 Can training on games improve LLM reasoning?

Introducing GlyphBench: 360+ game tasks for language-model RL.

We use it as a benchmark, a research tool, and a post-training suite.

🧵⬇️

[Vmax @VmaxAI]

10/ 🔎 How does an agent arrive at its next move?

Our replay UI brings observations, reasoning, actions, rewards, and memory into one view.

Step through a trajectory to inspect decisions and understand where an agent succeeds or gets stuck.

[Vmax @VmaxAI]

@BEBischof @HamelHusain 👏

[Goodfire @GoodfireAI]

We built cybersecurity monitors for Kimi K3 and GLM 5.3 — more accurate, 50x faster, & 50x cheaper than an optimized LLM judge.

External red-teaming from @farairesearch found that our monitor greatly reduces universal jailbreaks. 🧵

[Emiliano Penaloza @emilianopp_]

Come join us next week at our CoLM workshop:
Context Beyond the Window

We have some great speakers coming to speak on the opportunities, limitations and safety-considerations of current LLMs.

We will also be cohosting a social alongside Reflection!
https://t.co/XOGJfxq0P5

[Emiliano Penaloza @emilianopp_]

Want to train your SWE agents 2× faster 🏎️💨?

We introduce KV-streams, which, rather than flushing the KV cache on agentic compaction, we preserve it and keep generating. This avoids the re-prefill cost on both the inference engine and the trainer.

Unlike other works that assume a fixed compaction strategy, we show that KV-streams can work with any compaction method in agentic tasks.

Using what we learn from experiments in text-based games. We scale KV-streams to SWE tasks, where we match the performance of re-prefill compaction and full context in half the time 🚀

https://t.co/vZAxjXyO6E

🧵

[Roger Creus Castanyer @creus_roger]

https://t.co/mNGi0oJ98p

[Vmax @VmaxAI]

@henrytdowling @jurlycat 🫡

[Vmax @VmaxAI]

Nethack is a really underrated benchmark

[Roger Creus Castanyer @creus_roger]

astra playing NetHack:

- score: 48,978
- BALROG progress: 49.4%
- max dungeon depth: 17
- max experience level: 14

important note: this is a cherry-picked episode (best of 10), and Astra has been able to modify its own game-playing harness

no privileged information or unfair game modifications, but it uses tools such as pathfinding to previously visited landmarks and code-written trackers for previously observed entities. these offload a fair amount of low-level reasoning, while Astra remains responsible for choosing what to do at every step

imo, the ability of LLMs to autonomously create tools that amortize low-level control costs (and then interpret and use their outputs in-context) is remarkable, and one of the reasons i'm excited about LLM agents. like, we could run the same tool computations and... concatenate the outputs into a flat vector for an MLP... but it would still have to learn what those outputs mean and how to use them. with LLMs we get free hierarchies and natural separation of responsibilities...

anyway, i'm now running episodes with the default settings from BALROG for a more fair comparison

the auto-research programme continues. will report back!

full run below ⤵️

[Augustine Mavor-Parker @MavorParker]

Interesting details from @deepseek_ai on the benefits of scaling environment/task synthesis.

If you want to scale open-ended environment generation for your own use case, sign up to the @VmaxAI waitlist - link below