Cortex: Measurement-Gated Evolution and Pruning of Coding-Agent Skills and Rules
Source
arxiv.org
Author
Samuele Giampieri
Date
Why it matters
Harness edits that pass once may not help on repeat runs. Gating each added or deleted agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → or rule on repeated with/without comparisons keeps agent instructions from bloating.
Key takeaways · AI-distilled
In Cortex's experiments with Claude Haiku 4.5, auto-loading rules did most of the work. Skills helped only when offered alongside the files a task touched, and skills offered on every turn were rarely opened.
Cortex never adds or deletes agent harnessThe scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.Full definition → text the AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → did not actually use, so the author cautions the planted-bad-suggestion test shows it keeps no unused text, not that it detects harmful instructions.
Gains shrank with stronger models and real projects: under Claude Sonnet 5 a single run's harness added little, and on a real open-source repo where the agent was rarely corrected, Cortex found nothing to learn.
On held-out tasks the learned harnesses raised convention-following well above a no-rules baseline, left a deliberately unlearnable control area unchanged, and added no text read on every turn.
Terms in this piece · Glossary
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
articleHarness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven SystemsPaul Barbaste, Tristan Darrigol, Germain Vu, Tom Wiltberger
articleCode Understanding is a Bottleneck for Coding AgentsNishant Balepur, Kiran Tomlinson, Tobias Schnabel
videoCodex, Behind the Harness — Dominik Kundel, OpenAIAI Engineer