How Software Factories Improve Themselves — Suraj Gupta, Warp
Source
youtube.com
Author
AI Engineer
Date
Context
An automated development workflow can improve by proposing changes to its own instructions for people to review. In Warp's presentation, a separate outer-loop AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → watches the runs and feedback of an issue-triage agent, then opens a pull request, a proposed code change, updating that agent's agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → file.
This turns feedback into an inspectable revision: the instruction change is recorded in Git and reviewed by a human. The mechanism gives teams a concrete way to maintain agent behavior over time, with a record of what changed and an approval step before adopting the revision.
Key takeaways · AI-distilled
In Warp's setup, an outer-loop agent watches the triage agent's runs and feedback, then opens a pull request to update its skill, so every skill change is tracked in Git and reviewed by a human.
Warp's persistent memory is a versioned, traceable store of facts that lets a Sentry agent reuse a root cause it already found instead of rediscovering it, and it works across harnesses including Claude Code and Codex.
model routingSending each request to a model chosen by the difficulty of the task, rather than using one model for everything.Full definition → keeps teams from paying Opus prices for triage or simple CI fixes; they can use Warp's auto models or write their own routing rules.
Gupta says Warp's internal evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → found UI tasks run well on GLM, and routing evals are the next step for the feature.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
model routing — Sending each request to a model chosen by the difficulty of the task, rather than using one model for everything.
videoRethinking Claude Code Best Practices — Lydia Hallie, Anthropic
videoStop Shipping LLM Settings That Only Win on Paper - Rohan Pandey & Balaji Varadarajan, DigitalOcean
videoYour SKILL.md Is 70% Stuff the Model Already Knows - Nikolay Petrov, Intel
videoYour Spec Has Bugs. Catch Them Before Your Agent Codes. - Rajath Mahendrappa Kotyal, AWS
