AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → latency is dominated by output verbosity, turn count and inference speed, not tool execution. Pick models by time per task, not only by score.
Key takeaways · AI-distilled
AA computes time per task from evaluation tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → usage, representative output speeds and recorded tool execution time. Tool calls account for only about 12% of total time; verbosity, turns and inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → speed explain the rest.
Claude Opus 4.8 was the highest-scoring available model but took about 23 minutes per task, while GPT-5.5 (xhigh) ranked top 5 on Elo at about 11 minutes, roughly half the time.
GLM-5.2 sits on the Elo versus time Pareto frontier at 1261 Elo and 16.3 minutes per task, ahead of GPT-5.5 (xhigh) at 1159, and was the top open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → model, with MiniMax-M3 next at 1113.
AA estimates Claude Fable 5 would take about 28.5 minutes per task, based on about 91 output tokens per second, 3.1 minutes of tool time and about 139,000 output tokens per task measured while it was live.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
