For computer RL Environments the driver you use is more important than the model.
On CUA Driver Bench, we find that Haiku 5.5 w/cua-driver scores higher than GPT 6.1 Sol w/Codex Computer Use.
Pydantic AI is now a supported agent in @ScaleAILabs AgentEnv Framework.
Put it in a realistic environment (an inbox, a help desk), give it a task, and a verifier grades the result.
Change one ID to run a different agent on the same task.
https://t.co/k5aC7lCO7d
At @ScaleAILabs we created a high fidelity RL Environment of a trading desk in a real Texas battery grid.
GPT-6.1 Sol made $85,771 trading a real Texas battery. The battery's real operator made $20,647.
How far can a better harness take a lightweight model?
Using Agent Env Framework from @ScaleAILabs, we let Haiku's harness recursively improve across 3 games and put it head to head against Opus.
See more at: https://t.co/LZkKRqLYaS
Most phone agents are tested in simulated environments, but the real world is messier: carts persist, listings change hourly, and navigation varies.
At @ScaleAILabs, we ran agents on real phones with real apps on AgentEnv Framework.
See more at https://t.co/os4LJLyTPG
Tensorlake is now a built-in sandbox provider in @scale_AI 's AgentEnv Framework. An open-source framework for building RL environments.
Check out the docs and the demo to see how the Tensorlake sandbox hosts a full environment for a PortSim task.
https://t.co/7mfsIbMeNy