Anyone running multi-model routing in an agent or IDE loop is likely paying a hidden cache-invalidation tax on every switch; this piece puts numbers on it and gives a concrete mitigation — pin the model per session rather than route per turn.
An analysis of AI model routers (like those shipped by Cursor and Ramp) finds that prompt caches are model-specific, so mid-conversation model switches force a full-price re-read of the entire context — costing up to 11x a warm turn and scaling linearly with session size.
It also finds that routing to cheaper models often fails to save money once tokenizer differences and cache penalties are accounted for, and recommends pinning models at the session level, citing vLLM's session-aware routing which cut switches by 79% and eliminated tool-loop violations.
Transcript
It gets worse because the tax grows with how much context you're carrying. Double the session, and you double the cost of every switch inside it.
Cost per 1,000 prefix tokens barely moved across a six-fold change in session size:
https://t.co/A4qxW15rKx https://t.co/RHsL4Zapk9
