Intel / article

Optimizing Agent Reliability and Tool Calling Efficiency for Chip Design Using NVIDIA NeMo Gym

Source: developer.nvidia.com
Author: Tanya Lenz
Date: Published Oct 14, 2026

From the Daily Brief

NVIDIA published a case study of Siemens EDA tuning an agent for chip design work. The team used NeMo Gym verifiers to grade the agent's final artifacts against domain checks rather than reading transcripts. Those diagnostics pointed to specific failure modes, which Siemens addressed by rewriting the agent's skills and the descriptions of its MCP tools. The benchmark score rose from roughly 55% to 95%, and the average number of turns dropped from 155 to 53, a cut of about two thirds. The reported changes were to the harness text and tool documentation rather than to the agent's model. The method transfers to other engineering domains with checkable outputs: build a verifier for the artifact, use its failures to locate confusing tools or missing guidance, and fix those before reaching for a larger model. The numbers come from the vendor and its customer, so the size of the gain is specific to this setup.

Read the 2026-10-15 Brief →

Context

A chip-design became more reliable after engineers checked its final timing-and-power files and used failures to revise its instructions and tool descriptions. According to a post on NVIDIA's developer blog, co-written with Siemens engineers, the verifier rejected missing or invalid results and checked numerical values against references. This exposed mistakes that successful individual tool calls had missed. Case study.

The reported composite score rose from about 55% to 95%, while agent turns fell from 155 to 53. These were development runs on two logic cells across 36 operating-condition combinations, with the model and coding held fixed. The result supports improving the surrounding workflow through output verification, within that small tuning benchmark. Experimental scope.

Key takeaways · AI-distilled

The Liberty-file verifier multiplies four pass/fail gates (coverage of every cell and PVT corner, file validity, no new failures versus the reference, and integrity against reused or truncated values) by the fraction of values within tolerance, so any failed gate scores zero.
Tuning ran about 25 rollouts of 1.5 to 2 hours each with Claude Opus 4.8 in Claude Code held fixed, on two cells across 36 PVT corners from the open GF180MCU PDK. Only skills and tool descriptions changed.
The concrete tool fixes: simplify overlapping file and shell interfaces and define tool inputs with explicit JSON schemas. Output fell from about 81K to 29K alongside the drop in turns.
Final-artifact checks caught errors per-step checks missed, including missing cells and operating conditions, wrong slew thresholds, and timing or power values out of tolerance. Errors often produced plausible output rather than clean failures.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
MCP — The Model Context Protocol — an open standard that lets any AI assistant plug into any tool or data source without custom integration code.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Read the source developer.nvidia.com