← All IntelClip / OtherContext construction: size, flexibility, cost
From Codex, Behind the Harness — Dominik Kundel, OpenAI · ≈4:20
“we don't blast through your token budgets and throw in a bunch of unnecessary content”
“the more context you have in your in your context, the higher it is that you have contradicting information and it causes confusion for the model”
“we know you're cost sensitive, so cashability is important as well”
What’s in it
- Breaks down the three goals of context construction in an LLM harness
- Explains why bloated context confuses the model and wastes tokens
- Shows how caching keeps agent costs down as plugins scale
Clip transcript
the LM inference using responses. But, what happens in the actual harness? The first step, arguably one of the most important ones, is context construction. And during that, we care about three things quite a lot. The first one is size. We want to make sure that, you know, we don't blast through your token budgets and throw in a bunch of unnecessary content, but also the more context you have in your in your context, the higher it is that you have contradicting information and it causes confusion for the model. The other part is flexibility. We want to make sure that, regardless of how many or how little skills you're using, you have a great experience regardless of how many plugins and MCPs you install. And of course, we want to make sure that things are performing and cost we know you're cost sensitive, so cashability is important as well.
Comments
Sign in to comment.
Loading comments…