Rendering a scene early gives an AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → something concrete to inspect and revise. Artificial Analysis' AA-3D evaluation found that stronger models tended to look at images of their work earlier. Producing more tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → or taking more turns did not have the same clear relationship with the results.
The ranking comes from people choosing between rendered scenes built from the same brief. It measures visual preference under the supplied tools and time limits, rather than every aspect of software quality. The early-inspection finding is also observational: stronger models may both inspect sooner and build better scenes. It does not prove that simply adding an early screenshot will reproduce their advantage.
Key takeaways · AI-distilled
Each AA-3D agent gets a written brief and an offline sandboxAn isolated environment where AI-generated code or agent actions run without being able to touch anything real.Full definition → with Blender, Rapier physics, Python, a headless browser and CC0 assets, and has up to three hours or 250 turns to ship a three.js page; a recruited panel then votes blind on pairs.
Looking early matters: the top five first view a screenshot of their own scene after a median of 10 to 13 turns, and early inspection tracks Elo closely (Spearman 0.82); MiniMax-M3 views its scene 54 times per scene but rarely by turn 20 and ranks 19th.
Cost per scene ranges from $0.20 for GPT-6 Luna, which still ranks eighth, to $32.08 for Claude Fable 5.1; input is 69% of spend because agents resend context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → each turn, and output tokens and turn counts show no significant link to Elo.
Each Intelligence Index point is worth about 18 AA-3D Elo (Pearson 0.92), a closer fit than Terminal-Bench 4.0 or SciCode; GPT-6 Luna and Sol beat that trend by 141 and 93 Elo, while text-only GLM-5.3 falls 197 below.
Agents spend 82% of tool time on browser tests at 33 to 38 seconds each, top models read files and docs about half as often, and on-page control count tracks Elo closely (Pearson 0.90); detailed scenes win more votes, but mostly because stronger models build them.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
sandbox — An isolated environment where AI-generated code or agent actions run without being able to touch anything real.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
