How NVIDIA Factory Operations Blueprint Coordinates AI Agents to Investigate Production Issues

NVIDIA has published a reference design for putting an AI agent in charge of investigating problems on a factory floor. The Factory Operations Blueprint, called FOX, is a pattern for adding a "factory manager" agent on top of the systems a plant already runs, rather than replacing them. Developers connect existing machine databases, manufacturing execution systems and enterprise resource planning software through adapters, configure specialist agents for particular jobs, and set which decisions an operator has to review. FOX is offered through early access.

The post's main example is a surface-mount technology line, the largely automated process that places components on printed circuit boards. The chain starts when the automated optical inspection (AOI) station, a camera check after soldering, flags a possible defect. The manager agent retrieves the board's identifier, the affected component and the image. That identifier links the board to records from other stations, which matters because a defect found at one station may have started several steps before. A Visual Inspection Agent then compares the image against reference examples and approved criteria using NVIDIA's Cosmos model, given those examples at run time instead of being fine-tuned. For an acceptable variation, the agent returns evidence supporting continued production under factory policy; a confirmed defect moves on, and an inconclusive result goes to a person.

For a confirmed defect, a Fleet Management Agent sends the board to rework. NVIDIA demonstrates this only in a simulated factory built with Isaac Sim and ROS 2, the common robot software framework. A real deployment needs factory-specific robot integrations, and NVIDIA says initial deployments should require operator approval before any handling. A Root Cause Analysis Agent uses the board's history to pull placement records, test results and machine readings. For a suspected solder fault, it uses the board's entry and exit times at the reflow oven to select the matching oven readings, runs them through the Kumo Anomaly time-series model, and compares them with other runs and manuals. Its output is a PDF report with source records, suspect measurements, a likely cause and a recommended correction for the line manager to review.

Underneath, the manager is a language model (NVIDIA names a frontier option or its open Nemotron models), the NemoClaw agent harness, which handles tools, memory and delegation, and NVIDIA OpenShell, which isolates execution and enforces limits on the files, commands and network destinations the agent can reach. FOX can run on a single NVIDIA DGX Station. Production data stays inside the plant only if every model and service runs there too; with an externally hosted model, the developer has to assess what data leaves. NVIDIA's own table of responsibilities leaves the factory-specific work with the adopter: mapping identifiers, timestamps and units across systems, writing factory-specific inspection criteria, validating agent output, and connecting physical equipment.

The post gives no measured results for this circuit-board example. Its claims that agents reduce manual effort and downtime describe the design's goal. The numbers in the post concern Siemens, not the SMT walkthrough: a Siemens report's estimate that unplanned downtime costs an average large plant in the surveyed sectors $253 million a year, possible cost savings of "up to 20%" for agent-run rush-order planning at Siemens' Erlangen electronics factory, and a possible reduction of up to 40% in time to market from virtual production preparation. NVIDIA says the SMT example is separate from the Siemens planning work, and the post gives no method behind those figures.