← All IntelClip / OtherContext compaction for multi-model handoffs
From The State of Model Routing — NVIDIA, Cognition, OpenRouter · ≈11:14
“most of the context by default will only be going to like one model.”
“context compaction is something you already have to solve if you want to do like really long-running agents.”
What’s in it
- Explains why running multiple AI models on one task inflates token costs
- Shows a fix: only one model sees the full context, others get a summary
- Reuses long-running-agent context compaction techniques for multi-model setups
Clip transcript
needs, but not more? >> Yeah, absolutely. So, I I I think like the the context here is it's actually very easy to actually create a system that's more expensive as soon as you're running like, you know, multiple models together cuz oh, no, like, you know, this one file reading. Now, every every one of these models is now reading this one file reading. So, now you're you're you're being charged like three times as much. Um, the the the the trick that we we spent a lot of our time on is um, most of the context by default will only be going to like one model. So, like most of the context let's say will be going to the small model. But, the thing you need to then tune very well is okay, like maybe you still show like what files it's reading, maybe you show like the high-level thinking of what it's doing back to the main model. Maybe you have the small model um, you tune its ability to present the context back to the main model. Um, and actually a lot of these problems already have been well studied in many domains already like context compaction is something you already have to solve if you want to do like really long-running agents. And so, this problem of taking long context compacting it in a way that is now understandable is the one that you can also apply to this domain and just kind of give the compacted context back to
Comments
Sign in to comment.
Loading comments…