Clip / AI Agents

Lost-in-the-middle explains why big tool prompts fail

From The 100-Tool Agent Is a Trap - Sohail Shaikh & Ankush Rastogi, Prosodica · ≈5:19

Gives the mechanism behind the accuracy drop — schemas buried mid-prompt are paid for but not reliably used.

What’s in it

Gives the mechanism behind the accuracy drop — schemas buried mid-prompt are paid for but not reliably used.

Clip transcript

choosing from a small and relevant set. One reason the fat agent fails is because it's lost in the middle problem. Model pays stronger attention to the beginning and end of the long context. When hundreds of tool schemas are packed into the middle, the model does not reliably use them. So, we end up paying paying for a huge prompt, and that prompts makes the decisions even harder.