Verifier-driven tuning took a chip design agent from 55% to 95%
Source
developer.nvidia.com
Author
NVIDIA AI
Date
Why it matters
Shows how to improve an MCPThe Model Context Protocol — an open standard that lets any AI assistant plug into any tool or data source without custom integration code.Full definition →-tool agent by verifying final outputs with a zero-if-any-gate-fails reward and tuning skills and tool descriptions. One setup cut turns 66% and output tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → 64% while lifting the benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → score to about 95%.
Key takeaways · AI-distilled
The verifier multiplies four pass/fail gates (coverage, validity, relative validation against the reference's existing failures, and integrity) by the fraction of timing and power values within tolerance, so any failed gate zeroes the reward regardless of accuracy.
The integrity gate checks that the AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → actually ran characterization rather than reusing, truncating or reformatting existing values. The authors frame such checks as necessary to stop agents cheating on tasks graded by their final artifacts.
The gains came from changing skills and tool interfaces, not the model: with Claude Opus 4.8 in Claude Code held fixed across about 25 rollouts, the team simplified overlapping file and shell tools and gave tool inputs explicit JSON schemas.
Final-artifact checks caught errors that intermediate checks missed, including missing cells and operating conditions, wrong slew thresholds and out-of-tolerance values, because characterization mistakes tend to produce plausible outputs rather than clean failures.
To keep tuning tractable, each run used only 2 of the GF180MCU PDK's 229 standard cells across all 36 PVT corners, and a single rollout still took about 1.5 to 2 hours.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
MCP — The Model Context Protocol — an open standard that lets any AI assistant plug into any tool or data source without custom integration code.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
