Representation Affects Retrieval: A Case Study of Skill Discovery and Routing in a Multimodal Agent Harness
Source
Kevin Dela Rosa
Author
Kevin Dela Rosa
Date
Terms in this piece · Glossary
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
How a agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → is represented in the system promptThe standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.Full definition →, not just which skills exist, decides what the planner picks — and in-context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → selection is the small-N counterpart to embedding-based retrieval at scale.
