How to Run Isolated Tenant Kubernetes Clusters on Shared GPU Infrastructure
Source
Tanya Lenz
Author
Tanya Lenz
Date
Why it matters
If you're running shared GPU clusters across multiple teams, this walks through the concrete tradeoffs — CRD conflicts, RBAC scoping, and GPU budget partitioning — that determine whether you need full cluster isolation or can get away with a shared-cluster multi-tenancy model.
Key quotes
“Running a dedicated Kubernetes cluster per team often results in more isolation than an organization requires.”
Ekin Karabulut and Saiyam Pathak
“Challenges include conflicting CRD versions, overlapping RBAC, and no clean way to carve GPU capacity into team-level budgets. At a certain scale, teams might start asking for their own clusters just to regain autonomy.”
Ekin Karabulut and Saiyam Pathak
“The critical setting is setOwner: false . The KAI Scheduler pod-grouper walks ownership chains (Job → Pod, Deployment → ReplicaSet → Pod) to auto-group workloads. Disabling vCluster owner rewriting allows KAI Scheduler to see the real hierarchy.”
Ekin Karabulut and Saiyam Pathak
“Note that KAI Scheduler handles scheduling—which pods land on which GPU and in what proportion. It does not enforce GPU memory isolation at the hardware level when GPU sharing is used.”
Ekin Karabulut and Saiyam Pathak
“Together, KAI Scheduler and vCluster provide a dedicated cluster experience for three teams with zero waste on a single GPU. The answer isn’t always more GPUs, but better utilization of infrastructure.”
articleHow to Use AI Agents to Prepare 3D Scenes for Simulation
articleTranslating CUDA Tile Operations from Python to Rust Using Agentic AI
articleHow NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin
articleAccelerating Dropless MoE Training in JAX with NVIDIA Transformer Engine
