Quantifies fast progress in visual error-spotting and the open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → lag, informing whether vision agents can guide physical repair and assembly tasks.
Key takeaways · AI-distilled
Claude Opus 4.5 held the 28% best score in November 2025; OpenAI's GPT-6 Astra now leads at 80% and is also the fastest model tested, at a median 3 minutes per photo, two to ten times faster than earlier leaders.
The benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → has 60 photos across three IKEA builds of rising complexity: a shoe rack, a bed frame and a dresser estimated at 1.5, 3.5 and 5 hours for humans. Models get the assembly manual and tools to inspect the images.
To score, a model must flag every step with a mistake and describe it. Epoch made realistic errors, such as laminate facing inward, and often kept building past them to make them harder to spot. Human performance has not yet been measured.
Epoch frames the task as a proxy for visual repair work like fixing a car or household appliances, and notes few existing visual reasoning benchmarks test this agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition →.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
