How to Run Isolated Tenant Kubernetes Clusters on Shared GPU Infrastructure

Teams can have separate Kubernetes administration while sharing the same GPU hardware. An NVIDIA tutorial combines vCluster, which gives each team its own cluster control plane and permissions, with KAI Scheduler, which assigns GPU capacity through team quotas. The example runs three teams' workloads on one physical GPU.

The separation has a specific boundary. Independent control planes let teams manage their own cluster configuration, but fractional GPU scheduling does not enforce hardware-level GPU memory isolation. The tutorial targets trusted internal teams and says applications must respect their memory allocations. This is a way to share equipment while preserving administrative autonomy, with stronger separation required for untrusted tenants.

“Running a dedicated Kubernetes cluster per team often results in more isolation than an organization requires.”

Ekin Karabulut and Saiyam Pathak

“Challenges include conflicting CRD versions, overlapping RBAC, and no clean way to carve GPU capacity into team-level budgets. At a certain scale, teams might start asking for their own clusters just to regain autonomy.”

Ekin Karabulut and Saiyam Pathak

“The critical setting is setOwner: false . The KAI Scheduler pod-grouper walks ownership chains (Job → Pod, Deployment → ReplicaSet → Pod) to auto-group workloads. Disabling vCluster owner rewriting allows KAI Scheduler to see the real hierarchy.”

Ekin Karabulut and Saiyam Pathak

“Note that KAI Scheduler handles scheduling—which pods land on which GPU and in what proportion. It does not enforce GPU memory isolation at the hardware level when GPU sharing is used.”

Ekin Karabulut and Saiyam Pathak

“Together, KAI Scheduler and vCluster provide a dedicated cluster experience for three teams with zero waste on a single GPU. The answer isn’t always more GPUs, but better utilization of infrastructure.”

Ekin Karabulut and Saiyam Pathak