A model's benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → score may not reveal whether people can depend on it for sustained AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → work. In a May 26 outlook, Nathan Lambert proposes a practical test for open models: whether they become useful enough inside agent software, as Claude Opus 4.5 did in Claude Code in December 2025, to trigger an explosion in usage. Lambert argues that the robustness of the best closed models could keep the gap wider than benchmark comparisons suggest.
This is a dated forecast based on Lambert's own experience. Its useful contribution is the evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → criterion: judge an agent model by whether it reliably carries a workflow through, alongside its scores on individual tests.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
