Representation Affects Retrieval: A Case Study of Skill Discovery and Routing in a Multimodal Agent Harness
Source
arxiv.org
Author
Kevin Dela Rosa
Date
Why it matters
How a agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → is represented in the system promptThe standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.Full definition →, not just which skills exist, decides what the planner picks — and in-context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → selection is the small-N counterpart to embedding-based retrieval at scale.
Terms in this piece · Glossary
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
system prompt — The standing instructions a model receives before any user input — defining its role, rules, tools, and tone for the whole conversation.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
articleAn Empirical Study of Agent Skills' Downstream UtilityYu Cheng, Dehai Zhao, Zhongxin Liu, Qing Huang, Zhenchang Xing, Xiaoxue Ren
articleBeyond the Model: Demystifying Harness Effects in Software Engineering AgentsHaichuan Hu, Quanjun Zhang, Shengcheng Yu, Zhifei Chen, Tianyu Luo, Chunrong Fang, Zhenyu Chen, Liang Xiao
articleSubagents vs Agent Skills: Executing Reusable Knowledge for Long-Horizon Agentic TasksWasu Top Piriyakulkij, Rachel Lawrence, Alicia Curth, Sushrut Karmalkar, Niranjani Prasad