A model's listed price can mislead when estimating the cost of an AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →'s whole task. IBM researchers report running the same agent software across 417 AppWorld tasks: Anthropic's Sonnet 4.6 cost $79 in total, while OpenAI's GPT-4.1 cost $155, despite GPT-4.1's lower listed input and output prices. The figures come from their routing study account.
The authors attribute the reversal to caching: agent steps repeatedly reuse context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →, and Sonnet's cheaper cached-input charges changed the total bill. Selecting an economical model therefore requires measuring the workload's actual context reuse and execution costs.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
model routing — Sending each request to a model chosen by the difficulty of the task, rather than using one model for everything.
