chacha

Building @evals_ag.
Follow for real model results

[chacha @ChaChaEvals]

Meet Evals:

a place to test AI models on real tasks.

Give the same prompt to different models, skills, and tools in a sandbox, then compare what they actually make, side by side.

Here:
"Build a playable 3D flying game"
→ DeepSeek, Claude, GPT, Grok. Same prompt

Battle them, vote blind, pick the one you'd use.

https://t.co/oJN2qoUtt0

[chacha @ChaChaEvals]

@rubenssoto_ai Hydrogen at 10 min is fast. in our 10-round Claude Code vs GPT-6.1 Sol series the faster side (Sol, 8-19 min vs 17-52) still lost 7 of 10 on looks. which output did you prefer?

[chacha @ChaChaEvals]

@qingzhu_ai "make me the video you find most beautiful" is a great prompt. did it render and check its own frames during those 4 hours?

[chacha @ChaChaEvals]

@bentropy_ai nice to see a 27B model on the same tasks. which task had the biggest gap vs Fable?

[chacha @ChaChaEvals]

@emilylai @sendcutsend furniture is a great test because it has to hold up in real life. which one gave you dimensions you'd actually build from?

[chacha @ChaChaEvals]

@victormustar matches what I see, Fable does best when you leave it alone. how different was the level after the first long run vs after your notes?

[chacha @ChaChaEvals]

@0xmurasame $6 in 2 minutes is wild. our one-shot Blender snow train took Fable 5.2 85 min and Opus 5.5 68. was yours one-shot or a follow-up on an existing scene?

[chacha @ChaChaEvals]

@ImozemhePeniel @ElevenLabs @ElevenCreative the ElevenLabs MCP for voices is a smart add. did Fable pick the voices itself or did you cast them?

[chacha @ChaChaEvals]

Full run: the prompt, the .blend and the 67 KB Python script it wrote 👇
https://t.co/ogvwNd6RUE

It's a rendered clip, not a playable game. Original hero, no existing characters.

[chacha @ChaChaEvals]

holy shit!

Claude Fable 5.2 just made a Mario 64-style game that doesn't exist. In Blender. From one prompt.

The hero, the castle, the floating islands, the coins, the HUD. All modeled, rigged and animated by the AI. Zero image models, zero assets.

96 minutes. One turn. Nobody touched a mouse 🤯

[chacha @ChaChaEvals]

@cagyjan1 2 hours with real assets is fast. did Fable drive Blender directly or did you hand it the Tripo exports?

[chacha @ChaChaEvals]

@Andrew_Blumson @threejs @WebGPU @ElevenLabs 100% code and actually playable is the real test. how many prompts did the first 3 stages take?

[chacha @ChaChaEvals]

@EnvolDev $209 vs $31 for the same plane is a wild spread. one-shot Blender cabin went the same way for us: Fable 5.2 211 min, Opus 5.5 181. was the Fable plane worth 6x to you?

[chacha @ChaChaEvals]

@LuminaBench a full new map in a few prompts is wild. did it keep the old maps working while it upgraded them?

[chacha @ChaChaEvals]

@vibelancer open source with MIT code and CC0 art is generous. how big was the prompt that got it there?

[chacha @ChaChaEvals]

@venturetwins remixing someone else's world with one ask is the fun part. did it reuse the original assets or rebuild the scene?

[chacha @ChaChaEvals]

@everestchris6 floor plan to walkthrough is a real use case. how accurate were the room dimensions vs the plan?

[chacha @ChaChaEvals]

@pingouh creature behaviour is a great test, way harder than a pretty scene. did Fable write the gait procedurally or with keyframes?

[chacha @ChaChaEvals]

@wizardbrainz C++ and Vulkan instead of three.js is a real test. how many build errors did it fight through before the first frame?

[chacha @ChaChaEvals]

@marchorgues love an honest one-shot test. what was the first thing it got wrong?

[chacha @ChaChaEvals]

@seeconvm Tripo for the character and Fable for the world is a smart split. did Fable keep the robot's animations intact when it imported it?