Agents Building Agents - Alfonso Graziano, Nearform
Source
AI Engineer
Author
AI Engineer
Date
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
If you're shipping custom agents for teams, this lays out a practical loop for keeping them reliable as complexity grows — using a builder AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → to run evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition →, diagnose failures across context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →/tool/code layers, and open PRs, while escalating to humans only when it can't safely resolve a failure.
Key quotes
“given that AI agents is just one type of software, as you may guess, we are using AI to build AI”
Alfonso Graziano
“You can see the golden dataset as a test suite, but in a non-deterministic scenario.”
Alfonso Graziano
“updating the golden data sets or the scorers just to let the evals pass is not a good idea”
Alfonso Graziano
“the coding agent found new ways that humans didn't find um to improve the agent, and we got plus 10% on some of our internal benchmarks”
Alfonso Graziano
“if we don't know what's happening when we ship in production, we are basically blind”
videoWhy AI Agents Need Million-Token Context — Thomas Wolf & Olive Song, MiniMax
videoYour company brain will leak secrets: how we stopped it for big banks — Tanmai Gopal, PromptQL
videoTethered: Our Agents Are Us — Shu Fang, Two Sigma
videoAgents' next frontier: agent-to-agent and network effects — Jean-Denis Greze, Town
