We're proud to have collaborated on a new benchmark built by @GoogleAI: Physical-Engineering's Last Exam (PLE).
Physical engineers design the skyscrapers, power grids, and chip fabs we rely on, yet no existing benchmark tested whether AI agents could do that work end to end.
Together with Google and practicing engineers, we built 85 long-horizon tasks across mechanical, electrical, civil, and structural engineering. On the private test set, all frontier models score below 50%:
1. Claude Opus 5.5: 48.3%
2. GPT 6 Astra: 45.8%
3. Claude Sonnet 5.5: 43.5%
4. Gemini 4 Argon: 40.6%
5. Claude Opus 5: 40.0%
On the tasks we co-authored, the top agent satisfied 91% of individual constraints, but only 41.3% of its deliverable items passed every hard safety check.
Learning how AI fails at physical engineering can help us improve future models and build a safer world.
This benchmark is also an open competition. Bring your own agent setup and submit it for evaluation on our private test set at https://t.co/nOTWOUaV5v
Introducing SpaceChem Bench.
Players spent 15 years optimizing 2D molecular reactors in SpaceChem. We gave 16 frontier LLMs the game, 10 to 60 min per puzzle, and no archive access.
They found 20 new Pareto-frontier points on 11 of 55 puzzles, beating 11 community points.
Introducing TIS-100 Bench.
Players spent 10 years optimizing assembly programs in TIS-100. We gave 16 frontier LLMs the game, 10–45 min per puzzle, and no leaderboard.
They found 133 new Pareto-frontier points on 18 of 55 puzzles and broke 3 cycle records.
Highlights from the leaderboard:
- @AnthropicAI Opus 5.5 & @OpenAI GPT-6 Astra tied for #1 (55/55 solved, 11 puzzles with new frontier points; Opus found 72 points vs. Astra’s 32).
- @GoogleDeepMind Gemini 4 Argon placed #3 (8 puzzles extended), while Gemini 3.8 Flash solved 53/55 at the lowest cost.
- On Undecimal Twin Rounder, Opus 5.5 cut the 2,694-cycle human record to 1,918 (−29%)—and a community player built on Opus’s trick the next day to hit 1,744.
Huge thanks to @zachtronics for making an awesome game! Go grab it on Steam (https://t.co/OZrzxgiITU) or DRM-free on GOG (https://t.co/l0U0mPQnsg).
Introducing TIS-100 Bench.
Players spent 10 years optimizing assembly programs in TIS-100. We gave 16 frontier LLMs the game, 10–45 min per puzzle, and no leaderboard.
They found 133 new Pareto-frontier points on 18 of 55 puzzles and broke 3 cycle records.