Playground

Mapping the frontier of robot intelligence.

[Joel Jang @jang_yoel]

LLMs scoring high on sim benchmarks is great, but they usually get privileged info not available in the real-world. The comparison I am excited to see are frontier LLMs vs. frontier WAMs on real-world benchmarks, run by third parties. The optimal solution is probably a mix of both. Either way, rising zero-shot generalization numbers move the whole field forward.

[Playground @withplayground]

We’re very encouraged by Anthropic’s decision to report robotics eval results alongside its Fable release. We believe independent, real-robot evals will be vital as frontier labs begin to compete in robotics.

Congrats to the @AnthropicAI team on the release!

[Playground @withplayground]

Yesterday, Fable 5.2 became the first frontier model release to include a robotics benchmark result (64.9% on RoboDojo Sim). The problem is that simulation benchmarks systematically underrate certain model architectures.

[Playground @withplayground]

Our findings validate sim benchmarks as a tool for relative placements, but only in tandem with real-robot benchmarks. To help close the real-robot gap, we're:

• executing real-world runs of unseen LLM vs. VLA vs. WAM matchups on our hardware
• updating our methodology to weight real-robot results more heavily in our overall robot intelligence ranking
• publishing sim-to-real agreement by model family with every update to our intelligence index

Evaluate your model on our hardware: https://t.co/PznhPriNLA

[Playground @withplayground]

@preprintpigeon We appreciate the question! This reflects Anthropic's self-reported RoboDojo Sim score and not our official index result. We'll post real-robot results this week.

We acknowledge benchmark results have a wide spread and aggregate them to boost confidence in our relative rankings.

[Playground @withplayground]

Anthropic reports 64.9% for Fable 5.2 on RoboDojo Sim, which would rank it 2nd on RoboDojo's leaderboard (behind Q0-preview). To our knowledge, it's the first frontier launch to report a robotics benchmark.

Congrats @AnthropicAI! We look forward to testing it on real hardware.

[Claude @claudeai]

Fable 5.2 and Mythos 5.2 are a step above Opus 5.5 in almost all the capabilities we evaluated.

Testers have found that they communicate more clearly than past Fable and Mythos models. They generate outputs up to 30% faster than Fable 5.1 and Mythos 5.1.

[Playground @withplayground]

In weight-editing tests, Astra raised success rates from 68% to 88% on a simulated manipulation task by editing just 2,048 (0.00006%) of π0.5’s weights.

Our results suggest frontier LLMs may be able to write robot-control policies without collecting experience.

Full blog: https://t.co/B6BHan7bij

[Playground @withplayground]

We used GPT-6 Astra to hand-write a 19.8M-parameter neural network for a complex manipulation task that succeeded in 99/100 simulated trials.

Astra wrote the policy without internet, gradient descent, trial runs, or demonstrations.

[Playground @withplayground]

We're excited to announce the addition of RoboColosseum to the Robot Intelligence Index. It brings:

- Our first results on real humanoids
- 3x stronger connectivity in the real-robot ranking
- A first glimpse at capability gaps by embodiment

Congrats to @huggingface @BitRobotNetwork @eaic_sg
@DJiafei on the launch!

[RoboColosseum @RoboColosseum]

Introducing RoboColosseum

An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei

Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs

Thread ↓

[Playground @withplayground]

Full ranking and methodology: https://t.co/D6o22S5ffh

[Playground @withplayground]

FLUX 3 Action by Black Forest Labs debuts at #2 on the overall leaderboard.

World action models now occupy 2 of the top 4 overall slots, and the teams behind them rank as second and third overall labs, placing only behind OpenAI.

[RoboColosseum @RoboColosseum]

We’ve assembled a fleet of robots. This is just a part of it.

Now we put the models to the test, in the real world.

Coming online in 48 hours.

[Playground @withplayground]

@dreamlabs_ai @OpenAI @NVIDIARobotics Source and methodology: https://t.co/D6o22S5ffh

[Playground @withplayground]

On real robot benchmarks, world action models take every slot in the Robot Intelligence Index top 5.

GPT-6 Astra leads in the overall category where simulation performance is included, but WAMs continue to outperform LLMs in real-world tests.

[Freda Duan @FredaDuan]

10 Spicy Takes After Talking to 50 Friends at the Frontier of Robotics 🤖🔥

1/ Astra IS robotics' GPT-3 moment. Ironically, now that it's finally here, some in robotics seem reluctant to acknowledge it. Perhaps because it (sadly) didn't come from a robotics company?

2/ Robotics is becoming just another vertical for frontier labs. Not so different from coding in some respects. Why reinvent general intelligence for every industry?

3/ Robotics was too hard to underwrite until now. No convincing pretraining scaling law, no convergence on models, data, or hardware. No wonder some critics dismissed it as a fragmented industry playing an expensive "demo game". Astra may finally offer a credible path to scale.

4/ Robotics foundation-model startups have a case. But how strong is it? The argument: Astra handles System 2 (reasoning), while specialized models retain System 1 (skills) and System 0 (motor control). Fair. But isn't that already a narrower value proposition? And who says the boundary stops there?

5/ Follow the talent. Word is OpenAI reorganized its 30–40-person robotics team around the core Astra model. Hearing similar moves toward general models at @Tesla_Optimus. Org charts may tell us more than any demo.

6/ The robotics data gold rush is real. Million-hour procurement programs, some potentially reaching tens of millions of hours. Talk of budgets approaching $1B per frontier lab.

7/ Bullish on robotics hardware volumes. As intelligence becomes more accessible, the barriers to building useful robots could fall dramatically. The moat shifts toward reliability, dexterity, integration, and deployment.

8/ Robotics could fuel the next semiconductor and memory supercycle. A 100B-parameter model running on the edge requires ~50GB just for 4-bit weights. Now multiply that by millions of robots. Why do you think @JensenHuang is so obsessed with robotics? @elonmusk saw it coming: https://t.co/oKQi0XDhiL

9/ Is self-driving immune to the general-model threat? After coding and robotics, why assume self-driving remains the exclusive domain of proprietary, vertical models? Robotaxi increasingly feels like a solved problem to me. But if and when frontier labs move into self-driving, we could see an Apple vs. Android dynamic. Think about the implications for $TSLA, Waymo, and other OEMs.

10/ The next bottleneck may not be intelligence. It's deployment. Models are improving faster than our ability to deploy robots in the messy real world. Integration, hardware variance, failure recovery, and economics remain brutally hard. The real moat may be turning intelligence into reliable, profitable physical work.

--
I realize some of these takes may rub people the wrong way, especially those who have spent years doing the hard work of advancing robotics. That's not my intention.

I'm sharing them to spark debate, not to declare winners and losers. Astra has only been out for a month or so, and it's far too early to draw definitive conclusions. More importantly, when the TAM is this enormous - potentially the largest in tech - there should be plenty of room for different approaches and many winners.

++
Full diligence report: https://t.co/qlaR5qUzpr