- Home
- GPU Cluster Management Software
- KAI Scheduler
KAI Scheduler
What it is
KAI Scheduler is a free, open-source, self-hosted Kubernetes scheduler for allocating GPUs to AI workloads such as data processing, training, inference, interactive notebooks, and distributed jobs. Hierarchical queue trees provide quotas, limits, priorities, and over-quota borrowing between teams. Gang scheduling places distributed workload pods together when resources are available, while elastic workloads adjust replica counts within configured minimum and maximum values. GPU sharing supports time slicing, MPS, and MIG. The scheduler also offers topology-aware placement, time-based fairshare using historical GPU use, and priority-based preemption. Native Kubernetes integration is documented for Ray workloads, with Grove integration for complex workloads. KAI is designed for large clusters and supports cloud or on-premises infrastructure, including Karpenter autoscaling. Installation uses a production Helm chart, source build, or ArgoCD. A running Kubernetes cluster, Helm, and NVIDIA GPU Operator for GPU workloads are required. It is Apache-2.0 licensed and a CNCF sandbox project.
For and against
For
- Open-source Apache-2.0 project
- Hierarchical quotas and resource borrowing
- Gang and elastic scheduling
- GPU sharing with time slicing, MPS, and MIG
- Cloud and on-premises support
Against
- Requires a running Kubernetes cluster
- Requires Helm for installation
- GPU workloads require NVIDIA GPU Operator
- Self-hosted deployment
The verdict
KAI Scheduler provides detailed queueing and placement controls for shared GPU clusters. Its prerequisites and self-hosted model make it a fit for Kubernetes teams managing their own infrastructure.
Who it suits
KAI Scheduler suits teams running GPU-intensive AI workloads on Kubernetes. It fits organizations needing queue controls, shared GPUs, elastic jobs, or cloud and on-premises deployment.
On the record
- Cloud and on-premises
- The project describes compatibility with dynamic cloud infrastructure, including Karpenter autoscalers, and static on-premises deployments.github.com · read 7 Oct 2026
- Deployment
- The project provides production Helm installation, source build, and ArgoCD GitOps installation options for Kubernetes.github.com · read 7 Oct 2026
- Elastic workloads
- Elastic workloads can grow or shrink within configured minimum and maximum replica counts as capacity changes.kai-scheduler.dev · read 5 Oct 2026
- Fairness
- Time-based fairshare tracks historical GPU usage over a configurable window to distribute over-quota resources fairly over time.kai-scheduler.dev · read 7 Oct 2026
- Gang and elastic scheduling
- Gang scheduling places distributed workloads all at once, while elastic workloads can grow or shrink within minimum and maximum replica counts as capacity allows.kai-scheduler.dev · read 7 Oct 2026
- Gang scheduling
- Gang scheduling places all pods in a distributed workload together or waits until resources are available.github.com · read 5 Oct 2026
- GPU sharing
- GPU sharing supports time slicing, MPS, and MIG so workloads can share GPU devices.kai-scheduler.dev · read 7 Oct 2026
- GPU sharing limitation
- The GPU-sharing documentation says the default NonMemoryEnforced mode schedules fractional GPU workloads without KAI enforcing runtime GPU memory isolation.github.com · read 7 Oct 2026
- Hierarchical queues
- Multi-level queue trees support quotas, limits, and over-quota borrowing across teams.kai-scheduler.dev · read 7 Oct 2026
- Integrations
- The project states it is natively integrated with KubeRay for Ray workloads on Kubernetes and describes integrations with Grove and Dynamo for complex multi-component workloads.github.com · read 7 Oct 2026
- Intended workloads
- It supports workloads ranging from interactive notebooks to multi-node distributed training in the same cluster.kai-scheduler.dev · read 7 Oct 2026
- License
- The public GitHub repository identifies the project as open source and lists an Apache-2.0 license.github.com · read 7 Oct 2026
- License and project
- KAI Scheduler is an open-source Apache-2.0 project and a CNCF sandbox project.github.com · read 5 Oct 2026
- Prerequisite
- The installation guide says NVIDIA GPU-Operator is required to schedule workloads requesting GPU resources.github.com · read 5 Oct 2026
- Prerequisites
- The installation guide lists a running Kubernetes cluster and Helm CLI, and requires NVIDIA GPU Operator for workloads requesting GPU resources.github.com · read 7 Oct 2026
- Priority and preemption
- Per-queue and per-workload priorities let critical jobs preempt while non-critical jobs back off.kai-scheduler.dev · read 7 Oct 2026
- Purpose
- KAI Scheduler is a Kubernetes-native scheduler for AI workloads that optimizes GPU allocation across data processing, training, and inference while keeping resource shares fair across teams.kai-scheduler.dev · read 7 Oct 2026
- Queues
- Hierarchical queues support quotas, limits, priorities, and borrowing resources across teams.kai-scheduler.dev · read 5 Oct 2026
