ARC-AGI-2 A New Challenge for Frontier AI Reasoning Systems
Source
arcprize.org
Author
Greg Kamradt
Date
Context
A harder reasoning benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → is useful only if its puzzles remain meaningful and its results comparable. ARC Prize's ARC-AGI-2 overview describes tasks that require interpreting symbols, combining interacting rules and changing a rule's application with context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →. It also aims to make the public and hidden test sets similar in human difficulty.
The human-solvability claim has a specific meaning: the organizers retained tasks that at least two participants solved within two attempts each. It does not mean every participant solved every task. This selection process gives readers a more precise basis for interpreting the benchmark's human–AI comparison than the headline alone.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
