Intel / article

GPT‑6 Astra

Source: simonwillison.net
Date: Published Sep 3, 2026

Context

OpenAI's GPT-6 Astra produced substantially different ARC-AGI-3 results depending on the software managing its interaction with the test. The 's own evaluation reports 62.7% with its standard setup at maximum reasoning effort, versus 99.9% with a provider adapter at high effort.

The adapter preserves internal reasoning state between requests and compresses longer conversations so the model can reuse earlier work. These are different configurations, including different reasoning settings. The headline score therefore describes Astra operating within a particular system; the surrounding software is part of the result.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Read the source simonwillison.net