Validating GPU Platforms with Runtime AI Inventory via the NVIDIA DSX AI Factory

NVIDIA has added a running inventory of AI workloads to AI Cluster Runtime (AICR), its open-source tool for setting up GPU clusters on Kubernetes, the system that schedules containerized software across many machines. AICR turns a tested combination of drivers, operators and settings into a version-locked recipe, renders it into a deployment bundle, and later checks a live cluster against that recipe. An earlier NVIDIA post showed that side of the tool for the Dynamo inference stack. A recipe can prove the platform was built correctly, but it says nothing about what application teams deploy on top of it afterwards. In NVIDIA's write-up, Mark Chmarny points to reusable image tags such as :latest, under which two machines can run different code and a later download can fetch an image nobody reviewed.

The new component is k8s-aibom, an open-source controller created by Google. In namespaces an operator opts in, it reads each recognized AI workload's definition and writes an AI bill of materials: a structured record, in the CycloneDX 1.6 format, of the serving software, container image and model. In NVIDIA's example, a vLLM deployment produced a record naming vLLM as the runtime, the vllm/vllm-openai image and the Qwen2.5-0.5B-Instruct model, which an ordinary software inventory listing only the image would miss. Each fact carries a label (declared, inferred, unresolved or verified) and a pointer to its evidence, so a reviewer can tell a value the team wrote into its configuration, such as the model name passed to the server, from one the controller guessed, such as a runtime matched from the image name. Two stock recipes for H100 inference on Google Kubernetes Engine include the component by default, and other GKE recipes can turn it on.

NVIDIA is explicit that the record describes what was declared, not what ran. A record can report ready while its pod is still downloading the image, and it does not show that the workload was scheduled or succeeded. The observations themselves are unsigned, AICR leaves k8s-aibom's signature checks on model claims switched off by default, and comparing a recipe's intended setup with the workloads actually observed is listed as future work. The opt-in label limits which namespaces produce records, but the controller still reads workload definitions across the whole cluster, including container arguments and inline environment values. The added load is small in the upstream measurement NVIDIA cites, about four API requests a minute and roughly 30 MiB of memory, though that test ran with pending pods on CPU-only nodes and so measures the controller alone.