Agents Building Agents - Alfonso Graziano, Nearform

A coding agent can improve another agent through a repeatable experiment loop. Nearform's presentation describes establishing a baseline, proposing a specific explanation for failures, changing the target agent and rerunning its evaluations. Improvements are retained; regressions are rolled back, with reports preserving what was tried.

The test cases also evolve. The team groups failures found in recorded user interactions, asks domain experts to validate the groups, and turns accepted cases into regression tests. Human review matters because an unhappy user's report may describe intended behavior rather than a defect. This connects automated code changes to an expanding record of failures the team has actually investigated.

“given that AI agents is just one type of software, as you may guess, we are using AI to build AI”

Alfonso Graziano

“You can see the golden dataset as a test suite, but in a non-deterministic scenario.”

Alfonso Graziano

“updating the golden data sets or the scorers just to let the evals pass is not a good idea”

Alfonso Graziano

“the coding agent found new ways that humans didn't find um to improve the agent, and we got plus 10% on some of our internal benchmarks”

Alfonso Graziano

“if we don't know what's happening when we ship in production, we are basically blind”

Alfonso Graziano