Curious mind, passionate about research and how AI really works
AI / ML / Agents|正在进一步理解智能系统如何学习、推理与认识世界
中文 / English / 日本語
二次元・数学・绘画・音乐・钢琴
† vibe coding 重度依赖 †
@hosting997 Reverse engineering can involve cracking software or bypassing protections, and that could get your account flagged. Not worth risking your main account over it.
Artificial Analysis 推出了一个新基准:AA-3D
主要评估AI Agent 自主构建可交互的 3D 场景的能力。
目前前四名:
Sonnet 5.5:1194 Elo
Opus 5.5:1190 Elo
GPT-6 Astra:1165 Elo
Fable 5.1:1131 Elo
四款模型在统计上没有显著拉开差距。
不过能构建 3D 世界,不代表模型真正掌握了物理世界的运行规律。
这里面还有大量编程和工具调用能力的贡献。
未来如果再结合世界模型与实时生成技术,AI 驱动的虚拟世界会非常值得期待。
We have measured intelligence. Now it's time to see it.
We are launching AA-3D, our new benchmark for how well AI models build interactive 3D scenes. Each model works as an agent in an offline sandbox and turns a written brief into a three.js web page. A human panel of judges compares pairs of scenes built from the same brief without knowing which model made which.
AA-3D details:
➤ 55 written briefs across seven domains, including landmarks, machines, living bodies, games, and space. Each brief says what the scene must show and how people should interact with it, and leaves the rest to the model
➤ Each model gets up to three hours and 250 turns. Its sandbox has Blender, Rapier physics, Python, a headless browser to test its own work, and a CC0 library of materials, 3D models, and characters
➤ Panelists spend at least 30 seconds with each scene before they vote. One comparison in every four is a validation check
➤ Over 10,000 counted votes from over 1,000 panelists rank 21 models by Elo, with 95% confidence intervals
Key findings:
➤ Four models are statistically tied at the top: Claude Sonnet 5.5 (1194), Claude Opus 5.5 (1190), GPT-6 Astra (1165), and Claude Fable 5.1 (1131). The 95% confidence intervals of the other three overlap Claude Sonnet 5.5's
➤ AA-3D Elo rises with intelligence: each point on the Artificial Analysis Intelligence Index is worth about 18 Elo (Pearson 0.92)
➤ The best-placed open weights models, Kimi K3 (982) and MiMo-V2.6-Pro (977), rank 11th and 12th, more than 200 points behind first place
➤ A scene costs $0.20 to $32.08, depending on the model. The cheapest, GPT-6 Luna, ranks eighth
More results, more scenes, and how AA-3D works follow in the thread below.
Multi-Stream Attention Residuals
Multi-Stream Attention Residuals applies multi-head attention over network depth, achieving a 1.58x compute speedup over standard residuals at equal loss in tests on Mixture-of-Experts models up to 28B parameters.
https://t.co/FFelslP28s