@PrimeIntellect Are there benchmarks measure the impact of expert-per-gpu and prefill-decode disaggregation's impact on throughput that you could share? I'm curious how this compares with Slime's throughput
(1/n) Today, weโre releasing Cloning Bench.
Labs are paying 6-7 figures for clones of web apps to do web/computer use-based RL training.
At @VibrantLabsAI , our fundamental goal is to automate the creation of RL environments. For web/CUAs, one way that we do that is by using coding agents and custom harness to automatically generated the simulation environment.
We tested Codex, Gemini, Claude Code, and GLM using our harness on their ability to recreate a Slack workspace and benchmarked their performances.
We have published our methods, results and analysis here today: https://t.co/8t4AnGajrD
@realmcore_ When you say "subthreads.... return a compacted... trace to... main thread" What does this mean?
Is it similar to CC's compaction, or does it summarize every single action/tool-response? Don't you still risk polluting+using up the main thread's context?
@ariaurelium Could you please share what specifically gets irritating about using docker? We're starting work on setting up an deployable RL envs using docker and would love to learn about any issues you encountered.
@GenAI_is_real@lmsysorg Thanks. One more question: I know the focus of Miles will be to support the latest hardware, but will you also continue to support H100/H200 GPUs? These are the most widely available ones for renting rn.
@srush_nlp Thanks for the talk. One question I was left with was: why did you use micro-VMs instead of docker containers for the agent to generate rollouts?
@thdxr Looks good, but for Zen to be viable, new model availability can't be taking over a month after the model release...
That's a lot of time in LLM years.