Scale Labs

welcome to the lab.
from the researchers at @scale_AI

[Jason Wei @jaswwei]

1/ As agentic tasks grow longer and rollouts become more expensive, the choice between critic-based PPO and critic-free GRPO becomes increasingly important.

Critics can reduce gradient variance, allowing models to learn from fewer trajectories, but they also introduce additional compute costs and potential bias from bootstrapping.

Our new research @ScaleAILabs compares PPO and GRPO under a shared actor objective across 1,200+ training runs in control tasks, short-horizon math, and long-horizon terminal agents. We studied how three factors influence this tradeoff: batch size, critic reliance, and task complexity. 🧵

[Scale Labs @ScaleAILabs]

Full results: https://t.co/dr0ZJr8f1R

[Scale Labs @ScaleAILabs]

🥇 @AnthropicAI's Fable 5.2 takes the top spot on our Remote Labor Index with @CAIS, our leaderboard which compares AI agents' work head-to-head with work by human professionals on real-world remote tasks.

The model successfully automates 41.3% of projects, nearly doubling the performance of the next-best model Opus 5.5, a major leap in AI's ability to handle real-world work.

A closer look at where the biggest jumps are coming from 🧵

[Scale Labs @ScaleAILabs]

Fable 5.2 is also solving problems that were previously out of reach.

Nearly 50 of Fable 5.2's wins came from projects that none of the previous agents had successfully completed.

[Udari Madhushani Sehwag @UdariMadhu]

I am hiring a research scientist to work on safety and alignment. Come work with us. https://t.co/rjWFUMWKj9
@ScaleAILabs

[Scale Labs @ScaleAILabs]

@LeRobotHF Excited to see FineART dataset and FineART-VLA model powering language-guided robotics in LeRobot v0.7.0!

[Scale Labs @ScaleAILabs]

More results: https://t.co/x0VffeNzhF

[Scale Labs @ScaleAILabs]

Congrats to @thinkymachines on the release of Inkling 2 Small!

On agentic tasks, the model ranks 🥈 on SWE Atlas QnA 1.1 and performs on par with Kimi-K3 on the full SWE-Bench Pro V2 set and GPT-5.6 Terra on the hard subset. Notable results for a smaller model.

[Thinking Machines @thinkymachines]

Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor.

Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon.

https://t.co/VR02tw73fJ

[Scale Labs @ScaleAILabs]

@ShahriarGolchin @scale_AI ‼️

[Edgar Arakelyan @earakely]

At @ScaleAILabs we created a high fidelity RL Environment of a trading desk in a real Texas battery grid.

GPT-6.1 Sol made $85,771 trading a real Texas battery. The battery's real operator made $20,647.

[Anas Mahmoud @nas_mahmoud_]

The closest I ever got to @johnschulman2

[Scale Labs @ScaleAILabs]

@nas_mahmoud_ @johnschulman2 Let's go!! 🔥

[Daytona @daytonaio]

Daytona sandboxes are now built into AgentEnv, @scale_AI's open-source framework for RL environments.

Every environment and agent gets its own sandbox with Docker inside, so compose stacks, agent containers and image builds run unchanged.

Setup below 👇

[Scale Labs @ScaleAILabs]

Join the 100K+ people who have already downloaded the dataset to train multi-task policies, fine-tune the open checkpoint, and benchmark your models across 151 real-world robotics tasks.

- Dataset: https://t.co/cSXmun0uia
- Model: https://t.co/F1eBLbDF0f
- Training code: https://t.co/hiTVVKovnq
- Paper: https://t.co/1vcjKXEIqx

[Scale Labs @ScaleAILabs]

In collaboration with @HuggingFace, we’ve released the largest open robot dataset, alongside their first open System 1-System 2 model that can reason over step-by-step actions.

FineART is built with 40K+ demonstrations across 151 tasks and 500K subtask labels.

Together, the dataset and model were designed for each other, and both are available on @LeRobotHF where robotics developers work today.

[Boxd.sh @boxd_sh]

RL works. The problem is leaky environments.

If a rollout can see the last one's files or look the answer up online, the model learns to cheat. AgentEnv from @ScaleAILabs now gives each environment and agent its own boxd VM. It boots in 2 seconds and is deleted afterwards.

[Francis deSouza @fdesouza]

DualChem, a new benchmark from @scale_AI, tested 29 frontier models on dual-use chemistry questions: ones that could serve either legitimate scientific research or harmful ends.

They frequently flagged the risk, then answered anyway.

Recognition is not refusal.

DualChem is part of our research collaboration with @NIST’s Center for Advancing Innovation and Standards for Super Intelligence (CAISSI).

Governments need independent testing capacity they don’t have today.

@madialder covered the findings at FedScoop:
https://t.co/g94ZUZ5wuC

[Ying Liu @liuying04]

SWE-Bench Pro V2 has joined the Specialized Intelligence Index.

@ScaleAILabs built SWE-Bench Pro v2 around challenging tasks from applications, business services, and developer tools. Our private commercial cases also help assess performance on codebases that aren’t publicly available.

By joining the SII, we’re helping teams compare models against the demands of maintaining and extending software. These results give engineering teams a stronger basis for deciding which models to test on their own repositories.

[Scale @scale_AI]

From the beginning, Scale was built by people who love solving hard problems. That same ambition is driving what we build next.

Our ML Research Internship Program is accepting applications in SF and NYC. Work on frontier AI research alongside @ScaleAILabs researchers, publish your work, and help build the next generation of AI.

Apply now. https://t.co/WptkoyoTZe