Intel / article

NVIDIA Run:ai Is Now Open Source as Part of NVIDIA DSX AI Factory Platform

Source: developer.nvidia.com
Author: Tanya Lenz
Date: Published Oct 13, 2026

From the Daily Brief

NVIDIA has open-sourced the Run:ai platform as part of its DSX AI Factory work. The release includes KAI Scheduler for GPU-aware Kubernetes scheduling, a GPU Fractioning Operator that splits CUDA devices between workloads, quota management, and a workload contract, and NVIDIA says each piece can be adopted on its own. Nebius, Crusoe and Mirantis are among the partners integrating it. For teams running inference or agent workloads on shared GPU clusters, this removes the commercial license that previously gated these features and lets them inspect the scheduling logic directly. Fractional GPUs are most useful for many small models or bursty agent jobs that would otherwise leave whole devices idle.

Read the 2026-10-14 Brief →

Context

NVIDIA's Run:ai open-source release gives infrastructure teams a choice between adopting the complete GPU workload platform and integrating individual components. One useful distinction is between deciding who is entitled to computing capacity and deciding where a job runs. KAI Resource Management represents team hierarchies and quotas; KAI Scheduler handles placement of workloads on cluster resources.

That separation lets a team add a missing allocation or scheduling capability to an existing platform. The complete stack is also offered as an NGC Community Product without a paid Run:ai subscription, although that package has separate download terms and does not include enterprise support.

Key takeaways · AI-distilled

NVIDIA retired Run:ai's proprietary GPU fractioning in favor of native CUDA 13.4 fractions, where the driver enforces a hard memory cap per workload; the operator is a scheduler-agnostic Kubernetes layer plus a per-pod metrics exporter.
KAI Scheduler is Apache 2.0 and a CNCF project; it does all-or-nothing gang scheduling with in-memory simulation before placement to avoid needless evictions, and can run in an existing cluster without the rest of Run:ai.
Workload Map ships as a Go library that works offline on unstructured objects, describing workload types like PyTorchJob or RayCluster once so tools need not hardcode each CRD; it deliberately does not schedule, queue or autoscale.
Model Streamer, enabled in vLLM and SGLang with --load-format runai_streamer, loaded weights 3x to 6x faster than the default loader in Azure's published benchmarks on eight H100 VMs, at a steady 80 Gbps versus about 40 Gbps.
The NGC Community Product needs no paid Run:ai subscription but excludes NVIDIA enterprise support, and downloading it still requires accepting NVIDIA's software license terms.

Terms in this piece · Glossary

sandbox — An isolated environment where AI-generated code or agent actions run without being able to touch anything real.

Read the source developer.nvidia.com