Vibeleaderboard
← All Intel
Intel / article

Analyzing GPT-5.5 & Opus 4.7 with ARC-AGI-3

Source
arcprize.org
Author
Greg Kamradt
Date

Context

A successful level can hide an incorrect understanding of the game. ARC Prize's analysis of GPT-5.5 and Opus 4.7 runs describes agents carrying a mistaken rule into later levels even after an early win. Inspecting the sequence of actions helps distinguish a reusable strategy from success that the environment happened to permit.

The analysis has an important separation from the leaderboard result. Because the default GPT-5.5 tests did not return reasoning traces, its qualitative examination used an alternate setup that elicited more explicit descriptions. The official score came from the standard setup. The examples illustrate possible failure mechanisms, but their richer reasoning record should not be presented as the trace behind that official score.

Read the source arcprize.org
More from Greg Kamradt
Recommended reads
Comments

Checking sign-in…

Loading comments…