AutoWorldModel-Bench: A State-Centric Benchmark for Automated World-Model Research

AutoWorldModel-Bench asks coding agents to improve a model of how a game environment changes, without prescribing the improvement in advance. The benchmark supplies a starter model and a fixed computing budget across eight games. Agents receive structured game-state information directly, separating the prediction problem from the separate challenge of interpreting images.

The authors report improvements over the supplied starter in 63 of 64 sessions. This is evidence that the tested agents could find useful modifications within this bounded research setup. It evaluates experimentation with a provided model and explicit game states, rather than discovery of a general model of the physical world.

“World modeling is an unsettled field: architectures, training objectives, and state representations interact in complex ways, and no single recipe dominates across environments.”

Marjan Moodi et al.

“This makes it an ideal testbed for AI coding agents acting as autonomous researchers--a setting in which the improvement direction is not specified in advance, unlike the engineering-to-spec tasks that dominate current agent benchmarks.”

Marjan Moodi et al.

“Across 64 sessions, Codex-5.4 and Claude Opus 4.6 improve their starter on 63; in 91% of sessions the winning edit is a non-trivial research-style modification--a new objective, representation, rollout procedure, or architectural change--rather than a hyperparameter tweak.”

Marjan Moodi et al.

“Our benchmark offers a setting in which frontier coding agents can be evaluated on open-ended research rather than engineering-to-spec problems.”

Marjan Moodi et al.