The Loop Is the Product — Roland Gavrilescu, Introspection
Source
youtube.com
Author
AI Engineer
Date
Context
An AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →'s improvement process needs a durable record of what counts as good work. In his talk, Introspection co-founder Roland Gavrilescu proposes versioned agent recipes that preserve evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → rules, judging criteria, skills and human feedback independently of a particular model provider.
His talent-sourcing example connects that record to actual decisions: inspect the agent's activity for recurring patterns, compare automated judgments with human judgments, and test proposed changes with users before adopting them. The proposal gives teams a way to retain what they learn about a task as the models underneath their product change.
Key takeaways · AI-distilled
Gavrilescu argues that an agent's success depends on the quality of its signals and verifiers, and that each loop's output should feed the next one, which is why he calls the loop the product.
His second claim: 'system distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition →' is the moat. Evals, judges, skills and human judgment from each loop should be kept as portable, versioned 'agent recipes' you own, independent of any model provider. Introspection is releasing an early version, Pi recipes.
He proposes 'valued work per watt' as the score to optimize, and uses a talent-sourcing agent to show spotting patterns in traces, calibrating judges with a human in the loop, and A/B testing taste with real users before promoting a change.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
videoEvery Correction You Give Your Agent Is Data - Brian Douglas, Paper Compute
videoGive Your On-Call Its Own AI Teammate - Swaapnika Motapothula, Chinmay Gandhi & Vlad Tsvang, Meta
videoYour Agent Hates Your CLI - Ayush Bhardwaj, Allos AI
videoYou Can't Tell What an Agent Skill Will Do by Reading It - Roey Ben Chaim, Zenity
