ARC-Bench: Closed-Loop Replanning Masks Broken Action Ranking in Frozen JEPA World Models

A planning system can complete tasks even when its initial ranking of possible actions is poor. ARC-Bench examines models that favor an action when its predicted future looks closer to the goal in the model's internal numerical representation. The researchers tested whether that ordering actually matches the actions' true costs. Research abstract.

In the tested JEPA world models, they report substantial ranking failures across navigation and manipulation tasks. Frequent replanning concealed part of the weakness: reducing how often the system reconsidered its plan sharply reduced success. The finding supports evaluating the quality of proposed actions separately from whether repeated course corrections eventually finish a task.

“The assumption fails, severely and structurally: on the official manipulation audits the top-scored candidate is almost always suboptimal, and the same inversion appears in the maze domains.”

“Closed-loop success rates therefore systematically overstate the rankability of frozen latent representations.”