Shows how frontier models rank at agentic 3D scene building under blind human preference: the top four are statistically tied and open-weights models trail by over 200 Elo. Useful for picking a model for creative coding agents.
Key takeaways · AI-distilled
Each AA-3D AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → gets a written brief and an offline sandboxAn isolated environment where AI-generated code or agent actions run without being able to touch anything real.Full definition → with Blender, Rapier physics, Python, a headless browser and CC0 assets, and has up to three hours or 250 turns to ship a three.js page; a recruited panel then votes blind on pairs.
Looking early matters: the top five first view a screenshot of their own scene after a median of 10 to 13 turns, and early inspection tracks Elo closely (Spearman 0.82); MiniMax-M3 views its scene 54 times per scene but rarely by turn 20 and ranks 19th.
Cost per scene ranges from $0.20 for GPT-6 Luna, which still ranks eighth, to $32.08 for Claude Fable 5.1; input is 69% of spend because agents resend context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → each turn, and output tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → and turn counts show no significant link to Elo.
Each Intelligence Index point is worth about 18 AA-3D Elo (Pearson 0.92), a closer fit than Terminal-Bench 4.0 or SciCode; GPT-6 Luna and Sol beat that trend by 141 and 93 Elo, while text-only GLM-5.3 falls 197 below.
Agents spend 82% of tool time on browser tests at 33 to 38 seconds each, top models read files and docs about half as often, and on-page control count tracks Elo closely (Pearson 0.90); detailed scenes win more votes, but mostly because stronger models build them.
Terms in this piece · Glossary
sandbox — An isolated environment where AI-generated code or agent actions run without being able to touch anything real.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
