From 36% to 100%: How Self-Improving Agents Write Their Own Skills — Rafal Wilinski, Runlayer
Source
youtube.com
Author
AI Engineer
Date
Context
In an AI Engineer talk, Rafal Wilinski, a founding engineer at Runlayer, defines a agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → as a reusable playbook for an agent. Skills matter more as agents work longer, because a bad start can send an agent down an hours-long rabbit hole. The proposal is to deliver skills through Model Context Protocol (MCP)A published agreement for an AI application to discover what a separate program offers and ask it to run one of those offerings.Full definition →, a protocol for connecting agents to tools, from one governed ServerSoftware that receives requests, processes them and returns responses; the computer running it is its host.Full definition →, and to have agents write new skills by distilling successful and failed runs, an idea credited to the Voyager Minecraft paper.
The talk description reports that in one test a distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → skill took a task from 36 percent success to 100 percent. It does not say which task or how many trials, so it reads as a single illustration.
Key takeaways · AI-distilled
Wilinski argues skills matter more as agents run for hours, because a bad start can send an agent down an hours-long rabbit hole.
He names three problems holding skills back: they are built for developers, nobody has time to write them, and many clients do not support them.
The approach of distilling skills from agent traces is borrowed from the skill library in the Voyager Minecraft paper.
His case for the flywheel: frontier intelligence is rented, but distilled skills keep an organization's hard-won knowledge as models change.
Terms in this piece · Glossary
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
Model Context Protocol (MCP) — A published agreement for an AI application to discover what a separate program offers and ask it to run one of those offerings.
Server — Software that receives requests, processes them and returns responses; the computer running it is its host.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.