Support New AI Frameworks Faster Using NVIDIA Workload Map

NVIDIA Workload Map is an open-source way to describe the structure of AI workloads on Kubernetes once, so schedulers and management tools can understand a new framework without new code. On Kubernetes, a distributed training or inference job is a tree of resources: groups of pods, the units that run containers, each with a role, a replica count and a status. Each framework, such as JobSet, Ray or KServe, encodes that tree differently, so a tool has needed a separate adapter written in code for each one. NVIDIA's developer blog post says that makes support for every new framework a roadmap item, which is why it so often waits for a release.

Workload Map, formerly called Karta, moves that knowledge into a YAML definition that says where to find a workload's pod templates, replica counts and status, and how to translate its status into shared terms such as running, completed and failed. A Kubernetes resource type and a Go library read the definitions, and the project ships tested ones for 13 frameworks, including JobSet, PyTorchJob, RayJob, KServe and NVIDIA Dynamo. In the post's example, one JobSet definition lets a command-line tool show a job as one leader and two workers requesting 12 GPUs in total, instead of three Jobs and their pods listed separately. The same definition tells NVIDIA's KAI Scheduler which pods must be scheduled together or not at all, which it uses only for workload types without their own plugin, and lets NVIDIA Run:ai manage the job as one workload. Workload Map is Apache 2.0 licensed, and the evidence for its use is NVIDIA's own account: it says Run:ai uses it in production and that Elotl's Nova multi-cluster scheduler has adopted it.