Vibeleaderboard
← All Intel
Intel / post

Caching Cheaters on OpenRouter: when the cheapest token costs more

Source
x.com
Date
Tarun Chitra@tarunchitra

Every successful marketplace has winners and losers But some participants make it more of a dark forest than a chessboard We find some direct evidence for this — and some simple math to make these marketplaces more robust to rogue inference providers! https://t.co/Phgr7ktYdv

Context

On OpenRouter, a marketplace that sends each request for an model to one of several competing companies that run it, the provider with the lowest posted price is not always the cheapest one to use. A long post analyzing the marketplace explains why. Providers post three prices: one for new input text, one for output, and a much lower one for cached input, meaning text the provider has already processed and kept in memory. Agents resend the same instructions and history on every turn, so in work a large share of input should qualify for that cached rate. But the provider itself reports how many were cached, and the user has no independent way to check the number.

The authors argue that OpenRouter's routing rewards a price cut without checking the cache. Its documentation says a provider's chance of receiving a request rises with the inverse square of its price, so a small cut wins a disproportionately larger share of traffic. Watching which provider the router picked as cheapest, they found it chose the one with the lowest input price in all 64 cases where input and output prices pointed different ways, and the cached price had no detectable effect. A provider can therefore quote a low input price, attract more requests, report few or no cache hits, and bill repeated text at the full input rate.

Their paid experiments, which resent sequences of prompts to providers of Z.ai's GLM-5.2 model, found that pattern in practice. The three providers that most aggressively undercut Z.ai, which the post names as StreamLake, Novita and Baidu, showed worse cache performance and higher costs on repeated prompts, and cache hits fell sharply around the moments they changed their prices. For Novita, which the authors say repriced roughly 500 times in July, about five repeated prompts were enough for a user to pay more than they would have at Z.ai's own endpoint. The authors estimate the excess at around $3.5 million a year across OpenRouter's open-weight models, about 2% of its reported revenue, rising to an upper bound near $14 million if every provider behaved this way. Those figures are extrapolated from July and August 2026 data, and the post leaves the derivation to an appendix in a longer essay.

The claim is about cost, not proven cheating. In a note, the authors say that when they pinned requests to one of these providers with a fixed session ID, its cache recognized roughly 90% of repeated tokens; the failures clustered at price changes and switches between providers. They write that they cannot tell a deliberate cache wipe from a provider that rebuilds its systems whenever it changes prices, and that the user pays either way. Their proposed fix is for OpenRouter to rank providers by the price users actually pay, measuring each provider's cache-hit rate from real billing, so dropping the cache would raise a provider's effective price and cost it traffic. The post's author discloses investments in Together AI, Nous Research and Wafer, which participate in this market.

Terms in this piece · Glossary
  • open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
Recommended reads
Comments

Checking sign-in…

Loading comments…