NVIDIA Exemplar Cloud: Lessons for Unlocking Full Performance on AI Infrastructure

A healthy host-level benchmark can miss the configuration actually slowing a training job. In a B200 deployment described by NVIDIA, communication tests looked normal on the host, while operations that exchanged data between GPUs were two to four times slower inside the workload container. The container lacked both a network-topology configuration file and the setting pointing to it, so the communication library fell back to automatic detection. The case study identifies the missing runtime configuration as the problem. It shows why performance checks need to run through the same container and launch environment as the actual training workload.

“Two AI computing clusters built from identical NVIDIA H100, GB200 NVL72, or GB300 NVL72 systems can deliver materially different training throughput. We routinely see 8% to 12% gaps between partner deployments and the corresponding NVIDIA reference architecture (RA) on the same workload, same model, same global batch size.”

NVIDIA

“The CPU on the partner cluster was configured with C-states limited to C1 in BIOS. This is a common “low-latency” default that is actively wrong for AI training workloads.”

NVIDIA

“Allowing the idle cores to drop to C6 freed power headroom, enabling the busy cores to climb to 3.8 GHz and recover roughly 4% on this workload.”

NVIDIA

“Don’t increase QPS everywhere. QPS is fabric- and workload-dependent.”

NVIDIA

“If the path doesn’t resolve, NCCL fails silently with no error making this one of the harder gaps to diagnose without knowing where to look.”

NVIDIA