HAMi

HAMi is open-source GPU virtualization middleware for sharing and scheduling heterogeneous accelerators across AI workloads on Kubernetes. Workloads can request GPU memory and compute limits through Kubernetes resource controls. Scheduling options include binpack, spread, and numa-first, with binpack and spread selectable at node or GPU scope. The project lists accelerators from NVIDIA, AMD, AWS Neuron, Huawei Ascend, and other vendors; available isolation and partitioning capabilities vary by device. HAMi integrates with Kubernetes APIs, DRA, and CDI, and documents scheduling connections with Volcano, Kueue, Koordinator, and NVIDIA KAI Scheduler. A node metrics endpoint exposes device memory and core utilization for Prometheus scraping. Classic HAMi requires Kubernetes v1.23 or later. HAMi-DRA requires Kubernetes v1.34 or later, the DRA Consumable Capacity feature gate, CDI, and NVIDIA driver 440 or later. NVIDIA enforcement relies on user-space library interception, so applications that bypass CUDA are not covered. The project describes memory and compute enforcement as soft and best-effort, and recommends MIG when hardware-enforced isolation is required for compliance or SLAs. HAMi is free at no cost and is self-hosted.

HAMi provides Kubernetes-based accelerator sharing, quota controls, scheduling, and metrics across a broad device list. Device capabilities and isolation limits vary, so deployment requirements and enforcement constraints matter.

HAMi suits teams running AI, machine-learning, LLM, or HPC workloads on Kubernetes that need to share accelerators and control resource requests. It is aimed at operators comfortable with self-hosted Kubernetes deployments and the listed version requirements.

HAMi is a free, open-source layer for sharing accelerator capacity among Kubernetes workloads. It is best suited to teams running self-hosted AI, machine-learning or HPC clusters that need to allocate GPU resources across jobs. Its device breadth and quota controls are compelling, but software-mediated, best-effort enforcement makes it a poor choice when isolation must be hardware-backed.

Overview

Rather than reserving a whole GPU for every workload, HAMi lets teams request memory and core limits through Kubernetes resources such as nvidia.com/gpumem and nvidia.com/gpucores. This can help operators share accelerator capacity more finely; the practical benefit depends on the device, since isolation and partitioning capabilities vary across the supported-device matrix.

Placement policies include binpack, spread and numa-first. Binpack and spread can be applied at node or GPU scope, giving operators options for how workloads use available capacity. The built-in priority field has only two levels, however. Teams that need more nuanced queue priorities can integrate Volcano.

Key features

Accelerator sharing and scheduling

HAMi's supported-device matrix includes stable support for NVIDIA, AMD, AWS Neuron, Huawei Ascend, Cambricon, Enflame, Hygon, Iluvatar, Kunlunxin, MetaX, Moore Threads, Vastai and Biren devices. That makes it relevant to heterogeneous clusters, but the breadth should not be mistaken for uniform capability: some devices do not support memory isolation, core isolation or multi-card partitioning.

The project documents Kubernetes scheduling integrations with Volcano, Kueue, Koordinator and NVIDIA KAI Scheduler, as well as Kubernetes APIs, DRA and CDI. These connections make HAMi more adaptable to established cluster workflows, though they do not remove its deployment and device-specific requirements.

Metrics and enforcement

A node metrics endpoint exposes real-time device memory and core utilization for Prometheus to scrape. That gives cluster operators a way to observe accelerator use alongside their monitoring stack.

For NVIDIA devices, HAMi enforces limits through user-space library interception. Applications that bypass the CUDA library are outside that mechanism's coverage. The project characterizes vGPU memory and compute enforcement as soft and best-effort, and recommends MIG when compliance or SLAs require hardware-enforced isolation. Teams with those requirements should treat MIG, not HAMi alone, as the relevant safeguard.

Pricing

HAMi is free and open source: its HAMi plan costs free. There is no free trial because the software is offered as a free plan. This suits teams able to operate their own infrastructure; the deployment is self-hosted, rather than a paid hosted service.

Platforms

HAMi runs on Linux in a self-hosted Kubernetes environment. Classic HAMi supports Kubernetes v1.23 or later. HAMi-DRA requires Kubernetes v1.34 or later, the DRA Consumable Capacity feature gate, CDI and NVIDIA driver 440 or later. Installation uses Helm and requires Kubernetes, Helm, kubectl and installation permissions, so it is aimed at teams with cluster administration capability rather than users seeking a standalone desktop tool.

Who it's for

HAMi is a strong fit for platform teams sharing accelerators among Kubernetes-based LLM, machine-learning and HPC workloads, particularly when a cluster contains more than one accelerator family. Its quotas, scheduling choices and Prometheus-scrapable metrics address practical resource-management needs. It is not the right sole control for teams whose compliance or service commitments depend on hard isolation, and its two-level built-in priority field may be too limited for complex queues without an integration such as Volcano.

Pros and cons

• Pros: Fine-grained memory and core requests can help allocate accelerator capacity across workloads instead of dedicating whole devices.
• Pros: A broad stable-device matrix and integrations with several Kubernetes schedulers suit mixed-hardware clusters.
• Pros: Real-time memory and core utilization metrics can feed Prometheus, giving operators visibility into device use.
• Cons: Enforcement is best-effort, and NVIDIA applications that bypass CUDA library interception are not covered; this is a material constraint for strict isolation needs.
• Cons: Some supported devices lack memory isolation, core isolation or multi-card partitioning, so feature coverage is not consistent across hardware.
• Cons: The built-in priority field provides only two levels, limiting teams that need richer queue prioritization unless they integrate another scheduler.

Alternatives

GPUStack is another free, open-source GPU cluster manager; consider it when a cluster-management tool is the priority rather than HAMi's Kubernetes accelerator-sharing controls. Kueue is a free open-source Kubernetes project, making it an alternative for teams seeking that project rather than HAMi's device-sharing middleware.

dstack offers a free self-hosted orchestration stack and a GPU marketplace billed by usage, so it may suit teams looking for orchestration with a usage-based marketplace option. Backend.AI has a free open-source option for home users installing on their own hardware, plus a trial plan; it may suit a home user seeking that route.

ClearML has a free Community plan and describes its self-hosted version as open source, a possible fit for readers comparing self-hosted options. TORQUE Resource Manager is a free open-source project under the PBS license, while Apache YuniKorn is a free Apache-licensed project whose source must be built before use.

NVIDIA ShadowPlay is a free Windows app requiring Windows 10 or 11, a GeForce 551.52 driver or later, and a supported GPU; it is a different choice for readers seeking that Windows software rather than Kubernetes cluster middleware. For more options, see GPU Cluster Management Software.

Verdict

Choose HAMi if your team operates Kubernetes clusters and needs a free way to share and schedule heterogeneous accelerator capacity with quotas and utilization metrics. Its breadth and integrations are the main reasons to use it. Look elsewhere—or pair it with hardware-backed controls such as MIG—when hard isolation is essential, and plan on an additional scheduler integration if two priority levels are not enough.