dstack

dstack is an open-source orchestration layer for AI workloads across GPU clouds, Kubernetes, virtual machines, and bare-metal clusters. It provisions infrastructure and schedules jobs, with features including auto-scaling, port forwarding, and ingress. YAML files configure fleets, development environments, tasks, services, presets, and volumes. Supported accelerators include NVIDIA, AMD, TPU, and Tenstorrent. Documented backends include AWS, Azure, GCP, Kubernetes, GPU cloud providers, remote SSH hosts, and an experimental Slurm backend. Tasks can use frameworks such as accelerate, torchrun, Ray, and Spark. Resources can be managed through the CLI or HTTP API, and the server can run on a laptop or another environment that can reach the clusters in use. Service gateways can publish inference endpoints with HTTPS, custom domains, auto-scaling, and rate limits. The self-hosted open-source plan is free. Server data and project secrets are plaintext by default unless an administrator configures AES-256-GCM encryption. TPU support is limited to single-host instances with up to eight cores. GPU Marketplace prices vary by provider and are shown before provisioning; usage is billed against prepaid credits.

dstack combines workload scheduling, infrastructure provisioning, and service endpoints across several kinds of compute environments. Administrators should review the default plaintext storage and the stated TPU limit before choosing a deployment.

dstack suits teams managing AI workloads across cloud and on-premises infrastructure, including teams that need to provision resources and schedule jobs. Its documented workload types and accelerator support may fit varied compute environments.

dstack is open-source infrastructure software for teams orchestrating AI workloads across cloud and on-premises compute. It suits organizations working across different accelerators and environments; its breadth is a strength, but quota controls and TPU support are limited.

Overview

dstack brings provisioning and workload scheduling together across GPU clouds, Kubernetes, virtual machines, and bare-metal clusters. Workloads are configured in YAML and range from fleets and development environments to tasks, services, presets, and volumes. The server can run on a laptop or another environment able to reach the clusters and cloud accounts in use, making a hybrid setup possible.

Teams can manage resources with the CLI or use the HTTP API for direct server integrations and functionality beyond the CLI. Kubernetes is supported, and GPU utilization metrics are available. dstack does not provide quota controls, so teams that need enforced resource limits will need to manage that requirement elsewhere.

Key features

Scheduling across compute environments

dstack provisions infrastructure and schedules jobs across several kinds of compute rather than focusing on a single cloud or cluster type. Auto-scaling, port forwarding, and ingress support both jobs and services. Documented backends include AWS, Azure, GCP, Kubernetes, GPU cloud providers, remote SSH hosts, and an experimental Slurm backend. That range is useful for mixed infrastructure, though the experimental status of Slurm makes it a less settled choice for teams depending on that backend.

Accelerators and frameworks

Out-of-the-box accelerator support covers NVIDIA, AMD, TPU, and Tenstorrent hardware. The task guide names accelerate, torchrun, Ray, and Spark for distributed work, while the maker describes compatibility with any hardware, open-source tools, and frameworks. The TPU caveat is material: dstack supports single-host TPUs only, up to eight cores per instance. Teams needing larger or multi-host TPU deployments should look elsewhere.

Inference endpoints

Services can publish model inference as endpoints through gateways with HTTPS, custom domains, auto-scaling, and rate limits. Those controls give teams a path from scheduled workloads to managed endpoints without restricting dstack to batch jobs.

Security and support

Server data and project-scoped secrets are stored in plaintext by default. Administrators can configure AES-256-GCM encryption for stored data; because secrets share the default storage behavior, encryption is especially important where project credentials are involved. Documentation points users to GitHub for issues and Discord for questions.

Pricing

dstack OSS costs nothing and provides the open-source orchestration stack for self-hosting. It is the natural fit for teams willing to operate their own server and infrastructure; self-hosting also means they take responsibility for deployment and configuration.

dstack Sky GPU Marketplace has custom pricing: GPU usage is pay-as-you-go, charged per GPU-hour at rates that vary by GPU and provider. Compute is on-demand or spot, and usage is billed against prepaid credits; prices are shown in the console before provisioning. This suits teams that want marketplace GPU capacity, but costs depend on the selected resource and provider rather than a fixed subscription.

dstack Sky does not currently charge for BYOC mode. For a commercial offering, dstack Factory adds advanced multi-tenancy, usage metering, billing automation, and optimized inference presets for frontier open models; Factory has custom pricing.

Platforms

dstack supports API, Linux, macOS, self-hosted, web, and Windows. It is infrastructure software for managing compute, not a general-purpose consumer app.

Who it's for

dstack is best suited to AI infrastructure teams that need a shared workflow across cloud GPUs, Kubernetes, VMs, and bare-metal systems, or that want to schedule varied workloads through YAML. Its API and endpoint gateways also suit teams integrating orchestration into their own tooling or publishing inference services. It is a weaker fit when quota enforcement is essential, when default plaintext storage is unacceptable without administrator configuration, or when TPU workloads exceed the single-host, eight-core limit.

Pros and cons

Pros

• Broad infrastructure coverage: documented backends span major cloud providers, Kubernetes, GPU clouds, SSH hosts, and experimental Slurm.
• One workload model for varied work: YAML configurations cover fleets, development environments, tasks, services, presets, and volumes.
• Service gateway controls: HTTPS, custom domains, auto-scaling, and rate limits support published inference endpoints.
• Flexible entry points: the open-source stack is free to self-host, while Sky BYOC mode currently has no charge.

Cons

• No quota controls: teams cannot rely on dstack itself to enforce resource quotas.
• Plaintext defaults: server data and secrets remain plaintext unless server encryption is configured.
• Narrow TPU scope: support is limited to single-host instances with at most eight cores.
• Variable marketplace spend: Sky GPU rates depend on provider and GPU, and require prepaid credits.

Alternatives

Backend.AI is worth considering for teams comparing a freemium option with both a free self-installed home-user plan and a free trial. HAMi is an open-source GPU virtualization middleware option for AI workloads on Kubernetes. Kueue is a free open-source Kubernetes project. NVIDIA ShadowPlay is a free Windows app with a supported GeForce GPU and driver requirement, rather than a cross-environment orchestration stack.

ClearML offers a freemium model and describes its self-hosted version as fully open source. GPUStack is a free open-source GPU cluster manager. TORQUE Resource Manager is a free open-source project governed by the PBS license. Apache YuniKorn is a free Apache License 2.0 project whose source code must be built before use. For more options, browse the GPU Cluster Management Software list.

Verdict

Choose dstack if your team needs open-source orchestration that spans heterogeneous AI infrastructure and covers both scheduled workloads and inference endpoints. Its broad backend and accelerator support is the central reason to choose it. Look elsewhere if quota enforcement is a requirement, if plaintext defaults cannot be accepted, or if your TPU work exceeds its single-host limit.