Vmax

RL research focused on open-ended learning

[Vmax @VmaxAI]

Exciting work on unix environment generation led by @radbadgeoffbrad!

[Augustine Mavor-Parker @MavorParker]

The unix terminal is the natural interface for agents to get work done on a computer but how well can agents actually use unix?

Claude Code. Codex. Devin. Every frontier agent ships as a terminal tool.

With unix-ctf, Vmax is using setters and solvers to measure Unix competence.

[Vmax @VmaxAI]

@amit05prakash thank you!

[Vmax @VmaxAI]

@wwwjim @JPBrebner Have people found out what happens when you click on the flags?

[Vmax @VmaxAI]

@ivanburazin 🫡

[Vmax @VmaxAI]

@etpuisfume @MavorParker 🫡

[Vmax @VmaxAI]

@JPBrebner @wwwjim @wwwjim is the goat

[Vmax @VmaxAI]

Our designer @wwwjim has made something really special for this blog post.

[Augustine Mavor-Parker @MavorParker]

Blog post: https://t.co/afFtHONyAC
Paper: https://t.co/azjSZQbhwQ

[Augustine Mavor-Parker @MavorParker]

Vmax is building an open-ended learning system that generates and optimizes itself on tasks that it creates, avoiding human bias that may corrupt optimal learning curricula.

In PopuLoRA, we instantiate this as co-evolving populations of LLMs performing asymmetric self-play.

[Vmax @VmaxAI]

@creus_roger the multiple observation modes are really neat.

[Augustine Mavor-Parker @MavorParker]

We are so excited to have @tensorfi joining @VmaxAI!

Maxwill joins us from @Meta, where he was working RL and LLMs for recommendation. Previously, he has also worked @Tesla on the autopilot team and also in Quant finance at Kronos research. He also holds an MS in CS from Georgia Tech.

Maxwill simultaneously understands pre-LLM RL fundamentals but also how to scale pipelines for RL training for modern recommendation systems.

Maxwill is already levelling up our pipeline for automated environment design, pushing multiple PRs as soon as he joined.

Really excited about the velocity of his contributions and excited to share more soon.

[Vmax @VmaxAI]

Welcome Geoffrey!

[Augustine Mavor-Parker @MavorParker]

So excited to welcome Geoffrey Bradway as Member of Technical Staff @VmaxAI.

Geoffrey is a rare catch. He was an engineer at @GoogleDeepMind, Google for Youtube and also has experience in early stage companies, having been a previous @ycombinator founder and also VP of engineering at @numerai.

Fitting the Vmax DNA, he has experience with RL before it was cool (doing RL all the way back in 2014).

Outside of work, Geoffrey does some really cool art with robotic drawing machines.

Cannot wait to share more about what he is cooking

[Augustine Mavor-Parker @MavorParker]

PR review is one of the fast growing categories in AI for SWE, now you can benchmark agents on *real* PRs

[Augustine Mavor-Parker @MavorParker]

So excited to have @lorenz_wlf join @VmaxAI as a research fellow this spring!

At NeurIPS last year, we caught up with Lorenz, realised how aligned he is with our research vision and invited him to join us shortly after.

Lorenz comes from the @FAICDT1 programme at UCL (where I did my PhD also) and is supervised by @mircomusolesi.

Previously he worked on differential privacy and personalized recommender systems at Apple and did his undergrad in mathematics and statistics at Imperial College London.

Lorenz’s research focuses on RL, RLHF and modular continually learning RL agents. He has contributed to papers in ICLR, TMLR and AI STATS.

So excited for him to join us and accelerate our efforts on unsupervised environment design.

You can read Lorenz's research in the replies.

Much more to come.

[Vmax @VmaxAI]

@yanjo115 👀

[South Park Commons @southpkcommons]

22/ Reinforcement learning, but make it automated. @MavorParker & @matthewjsargent showed us how they’re generating long-horizon environments at @VmaxAI.

https://t.co/Cy4ki0wW90

[Vmax @VmaxAI]

@creus_roger @MavorParker 🫡💪

[Vmax @VmaxAI]

welcome Roger!

[Augustine Mavor-Parker @MavorParker]

@VmaxAI is excited to have @creus_roger joining us as a research fellow!

Roger is joining us from @Mila_Quebec where he works with @pcastr and @GlenBerseth.

Roger Creus Castanyer is a brilliant RL researcher working on exploration, credit assignment, and skill discovery.

He is also fresh off of a NeurIPS spotlight and a recently accepted paper to ICLR, you can find more of his research in the comments.

Roger is significantly accelerating our research on automated environment design - looking forward to sharing what he is cooking!

[Augustine Mavor-Parker @MavorParker]

@VmaxAI As an initial step in this direction, we have built on top of methods like SWE-smith and BugPilot, adding to the list of repo profiles built by the swe-bench community

[Augustine Mavor-Parker @MavorParker]

This is a preview of many more tasks to come for Ares!

[Augustine Mavor-Parker @MavorParker]

RL progress is bottlenecked by infra for training and evaluation. @VmaxAI is excited to be partnering @withmartian, generating environments for the Agentic Research and Evaluation (ARES) framework