Theory of Space: Models Struggle to Explore and Map

Most tests of spatial reasoning hand a model a fixed set of observations and ask questions about them. Theory of Space, a benchmark described on the Stanford SAIL blog, asks whether a model can gather those observations itself. An agent is placed in an unfamiliar, procedurally generated multi-room layout and must decide where to move, which way to turn and when it has seen enough. The benchmark tests three linked abilities: building a consistent map from partial views, revising it when the environment changes, and using it to answer questions about directions, locations and perspectives. Each task comes in a text version, with symbolic observations such as a chair being front-left and near, and a vision version, with first-person rendered images. That pairing lets the researchers tell reasoning failures apart from perception failures.

The benchmark's most distinctive feature is that the agent must write out its current map as structured data at every step. This lets the researchers check what the model believes, not only whether its final answers are right. According to the blog, performance drops when models have to explore. In the vision environments GPT-5.2 fell from 57.1% to 46.0% and Gemini 3 Pro from 60.5% to 57.3%, compared with being handed the observations, and the gap grew with environment complexity. Scripted agents reached the target coverage in about 9 steps, while the foundation models took 14 or more and still ended up with less accurate maps, often chasing each new door and leaving rooms half explored. The written-out maps show where vision falls short. Object orientation was correct only 20 to 32% of the time with images, against 91 to 92% with text. Beliefs about objects seen earlier also degraded over time, with stability scores of 56 to 62% in vision.

Revision was the weakest area. After the first exploration the researchers secretly moved objects, then checked whether the agent noticed and updated its map. With text observations GPT-5.2 identified orientation changes 97.9% of the time, but with images only 14.3%. In the vision setting GPT-5.2 also kept the old orientation 68.9% of the time even after seeing the object in its new position. The blog calls this belief inertia. These figures come from the blog's summary. The blog lists six models but gives headline numbers for only a few, and the full paper on the project page was not reviewed here. The finding complements VAGEN, another SAIL post, which tries to train vision agents to state and update exactly this kind of belief about their surroundings.