← All IntelClip / OtherSidekick architecture beats sub-agents for caching and swap flexibility
From The State of Model Routing — NVIDIA, Cognition, OpenRouter · ≈18:23
“we don't use sub agents. We use what we call a sidekick, which is um, one sub agent that continually has a running context.”
“it's 10 times cheaper on on all those cache tokens.”
“How can you RL model to also be good at collaboration?”
What’s in it
- Explains why typical sub-agent setups waste KV cache and money
- Introduces Cognition's 'sidekick' design that keeps context cached across agents
- Previews RL training methods to teach models to collaborate, not just execute
Clip transcript
to say. >> Yeah. >> [clears throat] >> One thing you said earlier is like, oh, you get the caching benefit from like the main line agent. Um, you actually can get the caching benefit from the side agent. And um, this is actually one of the key things we talked about with our Devin fusion launch is that um, you kind of are leaving a lot on the table if you do a main agent and sub agents type system. Um, so we don't use sub agents. We use what we call a sidekick, which is um, one sub agent that continually has a running context. So the main agent doesn't need to re-provide uh, context from earlier. Um, it's all still in the KB cache, right? Like it's 10 times cheaper on on all those cache tokens. Um, and then if you want to like switch the smart model to be like the one on the side or the one in charge, um it's actually totally fine, and you can kind of like do do the swapping back and forth. We're also spending a lot of time right now thinking about how do you train models to actually work collaboratively with other models? I actually I think there's a lot of literature out there on how you RL one model to do a task end-to-end. How can you RL model to also be good at collaboration? And when we think about it, we actually try both of these setups where let's RL the model being the orchestrator and the one deciding what gets delegated to other models, see how well that performs. Um and we also orchestrate it in a way where the model we're training is actually the executor, the sidekick, and see how well is it at executing other models' instructions. Um and we expect that to to be a probably a big lift in this next step of like multi-model orchestration is don't just like take models as they are and orchestrate them, but like can you actually co-design your models with the
Comments
Checking sign-in…
Loading comments…