Kubernetes v1.37 Gang Scheduling Hits Beta — and Brings a New API for AI/ML Workloads

Kubernetes v1.37’s workload-aware scheduling changes address a problem that has made AI/ML workloads particularly expensive to run on Kubernetes clusters: partial scheduling. A distributed training job that schedules half its pods burns GPU allocation without doing useful work. Gang scheduling — schedule everything or nothing — is the architectural answer, and it just graduated to beta.

What Changed

Gang Scheduling / PodGroup API (beta): The PodGroup API graduated from v1alpha2 to v1beta1. This is the foundational primitive: a PodGroup defines a minimum number of pods that must be scheduled together or not at all. If the cluster cannot accommodate the full group, no pods land. For batch workloads and distributed training jobs where a partial allocation is waste, this closes the loop between “pods scheduled” and “work running.”

CompositePodGroup API (new): A new API for expressing multi-level topology constraints, gang scheduling, and preemption policies in a single object. The notable addition here is native support for JobSet and LeaderWorkerSet (LWS) — the workload APIs that have become default for AI/ML job orchestration — without requiring extension APIs. CompositePodGroup handles hierarchical scheduling: a job with heterogeneous requirements (different node types, different resource classes) can express them in a single scheduling unit.

Workload-Aware Preemption (WAP) (beta): More nuanced control over which pods get evicted during resource contention. The alpha version introduced the concept; beta graduation means the API is stable enough to build on for production policy.

Shared DRA ResourceClaims for PodGroups (beta): Dynamic Resource Allocation resources (GPU partitions, RDMA devices, DPUs) can now be shared across pod groups. This matters for multi-tenant AI infrastructure where resource partitioning between workloads is tighter than node-level assignment.

workloadbuilder Library (new): Standardized building blocks for out-of-tree controllers to integrate with workload-aware scheduling. The Job controller is now natively integrated. Third-party controllers — custom operators for ML frameworks, batch systems — can use the same integration APIs rather than each implementing their own scheduling workarounds.

The Pattern

Stack Archive has tracked the Kubernetes v1.37 release cycle across three specific features: DRA extended resources graduating to GA, HPA scale-to-zero reaching beta, and pod certificate trust bundles. Workload-aware scheduling is the fourth v1.37 thread — and the one most directly targeted at AI/ML workload patterns. Taken together, v1.37 is more coherent as an AI infrastructure release than any individual feature announcement suggests.

What This Means for Platform Teams

Gang scheduling changes the failure mode of partial scheduling from “silent waste” to “explicit rejection.” A PodGroup that cannot be fully scheduled produces a clear status rather than a set of pods waiting in Pending state while consuming resource quotas. Platform teams get visibility into scheduling failures at the workload level, not the individual pod level.

The CompositePodGroup API is the more consequential addition for teams building on JobSet or LWS. The current workaround — assembling scheduling constraints across multiple objects and hoping the scheduler handles them coherently — produces the kind of subtle failures that are hard to debug under load. A single object expressing the full workload’s constraints removes a category of operational guesswork.

The workloadbuilder library matters for teams running custom operators. Any operator that has implemented its own scheduling integration to handle gang-scheduling semantics can evaluate migrating to the standardized library and deprecate the bespoke code.

Source: Kubernetes Blog

Content created with AI assistance and reviewed for accuracy.