Vibeleaderboard
← All Intel
Intel / post

Cache hit rates explain a 9x provider cost gap for agentic tasks

Source
x.com
Date
A
ArtificialAnlys@ArtificialAnlys

Cache hit rates and Cost per Task by provider are now live on Artificial Analysis: Cost per Task for GLM-5.3 (max) varies ~9x across providers, driven by a combination of cache hit rate and token pricing Agentic workloads resend the full conversation history every turn, so most input tokens are cacheable, and cache pricing and cache hit rates drive most of the gap in Cost per Task. We already use measured cache hit rates to calculate Cost per Task, and are now publishing them for each model endpoint we track. What's new: ➤ Cache hit rates are now visible for all first-party and third-party endpoints, measured by running multi-turn agentic coding tasks and reading cached tokens from each provider's usage response. Cache hit rate is the share of cacheable input tokens served from cache. ➤ Cost per Task is now available by provider for open weights models, calculated from each provider's input, output, cache hit and cache write prices, applied at its measured cache hit rate. To standardize across providers for a given model, we use the same input, output and cacheable token counts from the model's Intelligence Index run. Key takeaways: ➤ Cost per Task on GLM-5.3 (max) spans ~9x…

Read the full post on X
Why it matters

Agentic loops resend history every turn, so cache pricing and hit rate dominate cost. Provider choice for the same open model can change task cost about 9x, and you can now compare it with measured data.

Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
More from ArtificialAnlys
Recommended reads
Comments

Checking sign-in…

Loading comments…