The model version and computing budget belong beside the headline score. ARC Prize's original o3 evaluation tested a preview system at two spending levels, with higher computation producing a higher result. An update explicitly says the subsequently released o3 was a different version.
That means the preview result should not be transferred automatically to the released product. The report also says the tested system had trained on part of ARC's public training set and that its exact internal mechanisms remained uncertain. It establishes performance under the documented conditions; a broader claim about general intelligence or a particular architecture needs additional evidence.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
