Portal2Bench – Can AI agents solve Portal 2?

Portal2Bench, an independent benchmark by Kingston Yip, tests whether AI agents can solve the test chambers of the puzzle game Portal 2, and its early leaderboard has a wide gap between agents running in OpenAI's Codex and those running in Anthropic's Claude Code. GPT-6.1 Sol and GPT-6 Astra in Codex each solved 20 of 26 chambers and GPT-6.1 Terra solved 19, while Opus 5.5 in Claude Code solved 11 and Sonnet 5.5 solved 6. GPT-6 Luna in Codex solved none, and Fable 5.1 in Claude Code solved 3 of the 15 chambers it has tried so far (Portal2Bench).

The game is paused while the model thinks. The agent sends key presses and view turns and gets back screenshots and its position, with a notes folder but no shell, web access or game console. Each of the 26 chambers, drawn from chapters 2 to 5, is a fresh run that counts as solved only if the agent reaches the next map within 300 plans and $30 of API spending at list prices. If the agent dies, it respawns at the last checkpoint and is told only that it died. Among solved chambers, the median cost ranges from $0.74 for GPT-6.1 Terra to $13.12 for Fable 5.1.

The scores rank model-and-harness pairs rather than models alone: each model runs in its own agent harness at its highest effort setting, so the Codex and Claude Code results differ in both the model and the software around it. The page also says most chambers are still being run because of compute and usage limits, Gemini 4 Argon in Antigravity and Fable 5.2 have no results yet, and the research write-up is listed as coming soon. Until the runs and the write-up are finished, the gap is a provisional result from partial runs, not a settled ranking.