Extra work at answer time can mean two different things. In its 2024 progress report, ARC Prize describes one system that repeatedly generates and tests candidate programs, keeping promising ones for further variation. Another approach adapts a model through additional training while solving the task.
Both approaches use computation beyond a single answer from a fixed model, but they change different parts of the process: the pool of candidate solutions or the model itself. The report also distinguishes its public leaderboard's semi-private task set and relaxed computing rules from the main competition. Those conditions matter when comparing the reported results.
Terms in this piece · Glossary
test-time compute — Spending more computation when the model answers — thinking longer, trying multiple attempts — to buy accuracy without training a bigger model.
