Full Walkthrough: Writing & Using Skills — Nick Nisi and Zack Proser
- Source
- youtube.com
- Author
- AI Engineer
- Date
Context
An packages a recurring procedure so that a developer does not have to explain it from scratch each time. This workshop describes a short routing description, the working instructions and optional reference material or scripts. The description helps the decide when the procedure applies.
Keeping specialized references separate lets the agent load the material relevant to the current task. But additional instructions can also make performance worse. The presenters describe comparing tasks with and without a skill and finding that an overly prescriptive internal skill reduced their score. The useful test is whether the procedure improves representative work, including whether the agent selects and follows it when needed.
- agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
“without scripts, the AI is just speculating on what you mean when you say, "Go get the latest commits."”
“It's a misnomer that skills are only a single markdown file.”
Nick Nisi
“I was making things worse because I was overly prescribing what to do with next JS and cloud code was just inherently good at working with next JS”
Nick Nisi
“all of that context is gold. Like the conversation, especially what failed, especially what didn't go well, especially what was frustrating”
Zack Proser
“I think I did a CD in a directory recently. Otherwise it's been like probably six or eight months now.”
Zack Proser
videoBetter AI Models Won't Fix Your Engineering Team — Nicholas Arcolano, Jellyfish
videoWhy AI Writing Still Reads Like Slop — Louis-François Bouchard, Towards AI
videoHow to Test AI Agents When No Benchmark Exists — Nathanael Weill, Air Canada
video1,368 AI Failures and What They Teach About Evals — Vishal Srivastava & Tanmay Sah, New York Life
Checking sign-in…
Loading comments…