Artificial Analysis Makes Time per Task Its Headline API Speed Metric

Announcing the next version of Artificial Analysis API Performance Benchmarking: Time per Task is now our headline performance metric, measuring how long a model takes to complete a task rather than how fast it streams The update applies to the ~1,000 serverless endpoints we benchmark multiple times a day for speed, latency and caching. Key changes: ➤ Time per Task is now the headline metric to compare models: as workloads become more agentic, the time to complete a task matters more than speed per token. A model that is fast per token but verbose can take longer than a slower model that uses fewer tokens. Time per Task divides a model's output tokens per Intelligence Index task by its End-to-End Output Speed. ➤ End-to-End Output Speed covers the full request: speed is now measured from sending the request to receiving the final token. That includes the wait before the first token (prefill, and reasoning for models that hide it) as well as generation. Output tokens now come from each provider's API-reported counts instead of the o200k_base tokenizer, and we monitor for discrepancies in reported counts. ➤ Agentic workloads are now part of every measurement: we benchmark short,…