Intel / article

Aa Briefcase Time Per Task

Source: artificialanalysis.ai
Author: Artificial Analysis articles
Date: Published Jun 24, 2026

Context

Artificial Analysis added a speed measure to AA-Briefcase, the knowledge-work it launched a week earlier, in which models produce deliverables such as financial models, board presentations and design mock-ups inside realistic multi-week projects (artificialanalysis.ai). The new figure, average time per task, is not a stopwatch reading from one deployment. It is estimated from the each model used during evaluation, a representative output speed for that model, and the tool execution time recorded during the runs, so it reflects how much a model writes and how fast it is typically served.

The useful way to read the results is the Pareto frontier of AA-Briefcase Elo against time per task: the models for which no other model is both higher scoring and faster. Claude Opus 4.8 from Anthropic is the highest-scoring model still available but one of the slowest, at about 23 minutes per task. Several reasoning settings of OpenAI's GPT-5.5 (medium, high and xhigh) sit on the frontier, and GPT-5.5 (xhigh) ranks in the top five on Elo while taking about 11 minutes, roughly half of Opus 4.8's time. GLM-5.2 from Z.ai is also on the frontier: it scores 1261 Elo against GPT-5.5 (xhigh) at 1159, but takes 16.3 minutes. It is the best model on the benchmark, with MiniMax-M3 next at 1113.

Tool calls and their execution account for only about 12% of total time. The rest comes from how verbose a model is, how many turns it takes and how fast it generates. Claude Fable 5, which led the benchmark at launch, illustrates this: the article says that if it were still available it would likely take around 28.5 minutes per task, based on the roughly 91 output tokens per second, about 3.1 minutes of tool time and about 139,000 output tokens per task measured while it was live. The article does not say why Fable 5 is no longer available.

Key takeaways · AI-distilled

AA computes time per task from evaluation token usage, representative output speeds and recorded tool execution time. Tool calls account for only about 12% of total time; verbosity, turns and speed explain the rest.
Claude Opus 4.8 was the highest-scoring available model but took about 23 minutes per task, while GPT-5.5 (xhigh) ranked top 5 on Elo at about 11 minutes, roughly half the time.
GLM-5.2 sits on the Elo versus time Pareto frontier at 1261 Elo and 16.3 minutes per task, ahead of GPT-5.5 (xhigh) at 1159, and was the top open-weight model, with MiniMax-M3 next at 1113.
AA estimates Claude Fable 5 would take about 28.5 minutes per task, based on about 91 output tokens per second, 3.1 minutes of tool time and about 139,000 output tokens per task measured while it was live.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.

Read the source artificialanalysis.ai