Kubernetes Autoscaling at Scale: The Datadog Approach

Individual service autoscaling configuration is a solvable problem. Fleet-wide autoscaling — applying consistent, policy-based scaling across hundreds of workloads running across multiple clusters — is where most platform teams get stuck. The bottleneck is the same one that slows every platform capability adoption: the per-service handoff model, where each team needs to understand and manually configure the tooling for their own workloads.

Datadog’s approach to fleet-wide Kubernetes autoscaling eliminates that handoff model through three deployment paths that range from bulk UI activation to GitOps cluster profiles to AI-assisted manifest generation.

The Three Deployment Paths

In-App Bulk Activation

The centralized setup workflow in Datadog surfaces idle cost estimates and savings potential per workload. Platform teams can activate autoscaling for multiple workloads without writing or reviewing any YAML — the system deploys DatadogPodAutoscaler resources in bulk from the UI. For organizations that want to demonstrate initial value quickly and defer policy customization, this path gets autoscaling running across a fleet in a single session.

The idle cost estimate display is significant: it makes the value case visible to application teams without requiring them to understand autoscaling configuration. When a service shows an estimated $2,000/month in idle compute, the conversation about enabling autoscaling becomes straightforward.

GitOps Cluster Profiles

For organizations where infrastructure changes require code review and version control, Datadog’s DatadogPodAutoscalerClusterProfile custom resources define autoscaling policy once at the platform level. Namespaces are labeled to opt into a profile; service-level overrides apply additional labels when workload-specific behavior is needed. A single profile covers an entire namespace, reducing the surface area for configuration divergence across similar services.

According to Datadog’s documentation, this approach “preserves existing change control processes” — teams that require PR approval for infrastructure changes get autoscaling via the same Git workflow they already use, not a parallel configuration system that bypasses their governance controls.

AI-Assisted Manifest Generation

The third path targets existing workloads already running legacy autoscaling configurations — Horizontal Pod Autoscaler, Watermark Pod Autoscaler, or Vertical Pod Autoscaler. Bits AI Dev and the Datadog MCP Server can detect existing HPA/VPA configurations, translate them into DatadogPodAutoscaler configuration equivalents, and prepare the changes as draft pull requests. Platform teams review and merge rather than author from scratch.

This is where the AI assistance creates genuine leverage: translating existing configurations is tedious but straightforward work with low tolerance for error. Automating that translation and surfacing it as a reviewable diff removes the implementation bottleneck without removing human oversight of the change.

Vertical and Horizontal Scaling Strategy

The platform covers both dimensions of Kubernetes scaling:

Vertical resizing adjusts CPU and memory requests in-place without pod recreation, which addresses the most common root cause of wasted compute on stable workloads: resource requests that were set at service launch and never updated to reflect actual production behavior. A service that requests 4 CPU cores and consistently uses 0.8 is wasting three cores of capacity per pod — visible in cost, invisible in performance.

Horizontal pod autoscaling migrates from legacy HPA configurations to Datadog’s unified solution, which can incorporate Datadog metrics (not just CPU and memory) as scaling signals. For workloads where queue depth, custom business metrics, or external signals drive scaling decisions more accurately than resource utilization, this removes the limitation of standard HPA’s metric scope.

The autoscaling work complements the cost optimization framing visible in other Kubernetes rightsizing analyses: vertical resizing and horizontal scaling are the mechanisms; the question of whether teams trust them enough to enable them is a separate organizational challenge.

Source: Datadog

Content created with AI assistance and reviewed for accuracy.