Two interlocked claw pieces. 12 of 16 models failed.
GPT-6.2 Astra: interlock 71→0, 24 min, $10.24.
Claude Fable 5.2: still locked, 94 min, $14.62.
All 16 models ranked at the end.
Pick up an apple by the stem, five-finger hand, one attempt each.
GPT-6.2 Astra: lifted at 9.0 s, 10.5 min, $7.44.
Claude Fable 5.2: lifted at 8.7 s, 24 min, $3.76.
21-model leaderboard at the end.
Three cube moves with robot hands. Playback 2×.
GPT-6.2 Astra: 3/3, 18 min, $10.70.
Claude Fable 5.2: 3/3, 28 min, $6.05.
All 19 models ranked at the end.
Apple over a wall, onto a plate. Five starts each.
GPT-6.2 Astra: 5/5, 12.3 min, $7.98.
Claude Fable 5.2: 5/5, 17.3 min, $2.14.
All 15 models ranked at the end.
Apple over a wall, onto a plate. Five starts each.
Gemini 4 Carbon: 5/5, 17.5 min, $0.75.
Claude Fable 5.2: 5/5, 17.3 min, $2.14.
All 14 models ranked at the end.
Two interlocked claw pieces. 12 of 15 models failed.
Gemini 4 Carbon: interlock 71→0, 90 min, $5.72.
Claude Fable 5.2: still locked, 94 min, $14.62.
All 15 models ranked at the end.
Pick up an apple by the stem, five-finger hand, one attempt each.
Gemini 4 Carbon: lifted at 6.9 s, 33 min, $2.30.
Claude Fable 5.2: lifted at 8.7 s, 24 min, $3.76.
20-model leaderboard at the end.
Three cube moves with robot hands. Playback 2×.
Gemini 4 Carbon: 3/3, 64 min, $3.29.
Claude Fable 5.2: 3/3, 28 min, $6.05.
All 18 models ranked at the end.
Turn a Rubik's cube through three set moves with a five-finger robot hand. One attempt each. Both did all three.
Claude Fable 5.2: 28 min to build, $6.05.
GPT-6 Astra: 43 min, $15.97.
All 16 models we've run on it ranked at the end.
2× speed.
Two interlocked claw pieces: get them apart without breaking either. 12 of the 14 models we've run have failed it.
Claude Fable 5.2: still locked after 94 min, $14.62.
GPT-6 Astra: apart in 53 min, $14.36.
Pick up an apple by the stem, five-finger hand, one attempt each. Both lifted it.
Claude Fable 5.2: lifted at 8.7 s, 24 min to build, $3.76.
GPT-6 Astra: lifted at 13.9 s, 11 min, $3.48.
All 18 models we've run on it ranked at the end.
Carry an apple and set it on the plate, 5 random starts. Both landed all five.
Claude Fable 5.2: 17 min, $2.14.
GPT-6 Astra: 8 min to build, $2.68.
All 12 models we've run on it ranked at the end.
Spot, a lidded bin, three objects: open it, bin them, close the lid, stand clear. 20 random starts.
GPT-6 Astra: 20/20, 68 min, $30.20.
Kimi K3.1 Pro: 0/20. Bins all three on seed 0, never shuts the lid. 179 min, $32.09.
2× speed.
Pick up an apple by the stem, five-finger hand, one attempt each. Both lifted it.
Kimi K3.1 Pro: lifted at 6.8 s, 9 min to build, 34 cents.
GPT-6 Astra: lifted at 13.9 s, 11 min, $3.48.
All 17 models we've run on it ranked at the end.
Two robot arms, two interlocked claw pieces: get them apart without breaking either. 11 of 13 models we've run have failed it.
GPT-6 Astra: apart in 53 min, $14.36.
Kimi K3.1 Pro: still locked after 101 min, $13.45.
DeepSeek V4.1 Flash 1017 vs 4 models: carry an apple over a wall and set it on the plate, 5 random starts.
GPT-6 Astra, Haiku 5.5, Gemini 4 Argon, Opus 5.5, one after another. Everyone lands it. DeepSeek does it for 7 cents.
2× speed.
Six models, one apple, one rule: pick it up by the stem. Same prompt, same five-finger hand, one attempt each.
Three rounds, then where all 15 models we’ve run on it landed.
Gemini 4 Argon lifted first at 4.7 s. Grok 4.7 never got it off the table.
Three face turns on a Rubik’s cube, five-finger robot hand. Both landed all three.
GPT-6 Astra: 43 min to build, last turn at 49.8 s, $15.97.
Claude Opus 5.5: 32 min, last turn at 30.8 s, $4.75.
All 14 models we’ve run on this task ranked at the end.
Two robot arms, two interlocked claw pieces, one rule: get them apart without breaking either. 9 of 11 models we’ve run have failed this one.
GPT-6.1 Sol: apart in 61 min, $1.63.
Claude Opus 5.5: still locked after 93 min, $17.39.
Same cube, same three moves, the smallest Claude.
Haiku 5.5 got the first turn at 4.2 s, then the cube slipped out of the hand. 71 minutes of trying, $2.45.
Opus 5.5, left, for scale: all three turns, 30.8 s.
Three face turns on a Rubik’s cube, five-finger robot hand. Both Claude models landed all three.
Opus 5.5: 32 min to build, last turn at 30.8 s, $4.75.
Sonnet 5.5: 101 min to build, last turn at 47.6 s, $18.19.
3× longer, 4× the cost.