Kubernetes Cost Optimization: Right-Sizing, Spot Instances, and Cluster Autoscaling Strategies

Most Kubernetes clusters cost more than they should. The gap between what workloads actually need and what operators request for them is the single largest source of waste. On a cluster with 50 services, a 30% over-provisioning on resource requests compounded across hundreds of pods adds up to a meaningful fraction of your infrastructure budget, often without anyone noticing because the cluster “looks healthy.”

This guide covers the specific mechanics of reducing that gap: how to use real usage data to right-size requests, how to shift baseline compute to spot instances without introducing flakiness, how to choose between cluster autoscaler and Karpenter, and how to make costs visible at the namespace and team level. Every section includes the tradeoffs, because cost and reliability pull in opposite directions and the right balance depends on your workload.

Why Resource Requests Are the Root Problem

Before going to spot instances or autoscalers, fix resource requests. They drive every downstream cost and scheduling decision.

The cluster scheduler uses requests to determine where a pod can run. The limits determine when a pod gets throttled (CPU) or killed (memory). If your requests are too high relative to actual usage, the scheduler reserves capacity that is never consumed. Nodes fill up on paper while being underutilized in practice, and the cluster autoscaler provisions new nodes to schedule pods that could have fit on existing ones.

The common pattern is padding: a developer sets cpu: 500m because it sounds safe, actual usage is 80m during normal operation and 200m at peak. Multiply that across 40 replicas and you have reserved 20 CPU cores that are 84% idle.

Using VPA Recommendations Without Auto Mode

The Vertical Pod Autoscaler in Off mode is a recommendation engine. It does not change anything in your cluster. Deploy it, wait two weeks for it to observe actual usage, then query its recommendations.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: payments-api
  namespace: production
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: payments-api
  updatePolicy:
    updateMode: "Off"
  resourcePolicy:
    containerPolicies:
      - containerName: api
        minAllowed:
          cpu: 50m
          memory: 64Mi
        maxAllowed:
          cpu: 4
          memory: 4Gi

After two weeks, inspect the recommendations:

kubectl get vpa payments-api -n production -o json | \
  jq '.status.recommendation.containerRecommendations[] |
    {container: .containerName,
     target: .target,
     lowerBound: .lowerBound,
     upperBound: .upperBound}'

The output gives you three values: lowerBound (safe minimum for most traffic), target (what VPA would set), and upperBound (headroom for spikes). The right-sizing strategy for production workloads is to set requests at the target value and limits at 1.5-2x target. Do not blindly set limits equal to requests unless your workload truly has a flat resource profile.

One thing VPA recommendations get wrong: they do not understand your deployment topology. If you run 3 replicas of a service, the total headroom across all replicas matters more than per-pod limits. A pod that can burst to use another pod’s slack via bin-packing is fine. A pod that routinely hits its memory limit and gets OOMKilled is not.

The Right-Sizing Process in Practice

Run this at the namespace level to get a view of waste across your cluster. The Kubernetes Metrics Server exposes current usage; compare it against configured requests:

kubectl top pods -n production --containers | \
  awk 'NR>1 {print $1, $2, $3, $4}' | \
  sort -k3 -rn | head -20

Then cross-reference with configured requests using kubectl get pods -n production -o json piped through jq. The pods where current CPU usage is consistently below 20% of requests are your first targets.

Do not right-size in one shot. Change one workload at a time, watch it for 48 hours, then move to the next. VPA Initial mode is useful for this: it applies new requests when pods are recreated (deployments, rollouts, evictions) but does not evict pods proactively. You get the benefit without the risk of forced restarts.

Spot and Preemptible Instances: The High-Leverage Lever

Spot instances (AWS) and preemptible VMs (GCP) offer 60-90% discounts over on-demand pricing. That discount is real. The risk is real too: the cloud provider can reclaim the instance with a two-minute warning. The engineering problem is: how do you take the savings without making your cluster unreliable?

The answer is segregated node pools combined with workload classification.

Node Pool Architecture

Run two node pools:

1. On-demand pool: small, reserved for control-plane components, stateful workloads, and anything that cannot tolerate a 2-minute eviction notice. Target 20-30% of your total compute here.
2. Spot pool: large, for stateless services, batch jobs, background workers. Target 70-80% of your compute here.

Label nodes to reflect their lifecycle:

# Node labels applied via node pool config or nodeSelector in your cloud provider
node.kubernetes.io/lifecycle: spot       # AWS standard label for spot nodes
cloud.google.com/gke-spot: "true"        # GCP equivalent

Use taints on spot nodes to force explicit opt-in:

# Taint applied to all spot nodes
- key: "spot-instance"
  value: "true"
  effect: "NoSchedule"

Deployments that can run on spot nodes must tolerate the taint and use a node affinity that prefers spot but falls back to on-demand:

spec:
  tolerations:
    - key: "spot-instance"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 80
          preference:
            matchExpressions:
              - key: "node.kubernetes.io/lifecycle"
                operator: In
                values: ["spot"]
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: "kubernetes.io/os"
                operator: In
                values: ["linux"]

The preferredDuringScheduling (weight 80) means the scheduler strongly prefers spot nodes but will place the pod on on-demand nodes if no spot capacity is available. This is the fallback behavior you want.

Pod Disruption Budgets: The Reliability Anchor

A PodDisruptionBudget (PDB) tells Kubernetes how many pods of a deployment can be voluntarily evicted at once. Without one, a spot reclamation event can take down all replicas of a service simultaneously if they happen to be on the same node being drained.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: payments-api-pdb
  namespace: production
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: payments-api

minAvailable: 2 means at least 2 replicas must remain healthy during any voluntary disruption. Set this based on your minimum viable capacity, not your normal replica count. If your service needs at least 2 replicas to handle traffic, set minAvailable: 2. If you run 10 replicas but 3 is the minimum, minAvailable: 3 gives you more flexibility during node drains.

Graceful Drain on Spot Reclamation

When a spot node gets a termination notice, the cloud provider gives you 2 minutes (AWS) before the instance is stopped. The node.lifecycle/spot-interruption condition is set on the node, and a well-configured cluster drains it gracefully. AWS Node Termination Handler (for EKS) and GCP’s similar tooling watch for this signal and trigger kubectl drain before the instance dies.

Your workloads need to handle SIGTERM properly for this to work. Pods that ignore SIGTERM or take longer than terminationGracePeriodSeconds to shut down will be killed mid-request. The terminationGracePeriodSeconds value should be longer than your P99 request duration plus your in-flight processing time.

spec:
  terminationGracePeriodSeconds: 60
  containers:
    - name: api
      lifecycle:
        preStop:
          exec:
            command: ["/bin/sh", "-c", "sleep 5"]

The preStop sleep gives the load balancer time to deregister the pod before the container starts refusing connections. Without it, you get a brief window where the load balancer still routes to a pod that is shutting down.

Cluster Autoscaler vs Karpenter

Node provisioning is where cost decisions compound. Both cluster autoscaler and Karpenter provision and deprovision nodes based on pod scheduling pressure, but their models are different enough that the choice matters.

Dimension	Cluster Autoscaler	Karpenter
Node selection model	Chooses from pre-defined node groups/pools	Directly provisions any instance type that fits demand
Provisioning latency	2-4 minutes (node group based)	30-60 seconds (direct EC2/cloud API calls)
Bin-packing	Conservative, prefers existing nodes	Aggressive, can consolidate nodes in-flight
Spot diversification	Requires multiple node groups per instance type	Native: bids across dozens of instance types automatically
Configuration surface	Node groups, min/max, scale-down delay	NodePool, NodeClass, disruption budgets
Multi-architecture	Manual node group per arch	Native ARM64 + x86 mixing
Maturity	GA, battle-tested, wide ecosystem support	GA since 1.0 (2023), growing fast
AWS-specific	Agnostic (works on GKE, AKS, EKS)	AWS-native (EC2NodeClass), Azure preview

The key advantage of Karpenter is spot diversification. Cluster autoscaler requires you to pre-define node groups for each instance family you want spot instances from. To get good spot availability, you need 4-6 node groups per region. Karpenter handles this in a single NodePool by expressing capacity in terms of workload requirements and letting it pick from dozens of compatible instance types.

A Karpenter NodePool for spot workloads:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot-general
spec:
  template:
    metadata:
      labels:
        node.kubernetes.io/lifecycle: spot
    spec:
      taints:
        - key: spot-instance
          value: "true"
          effect: NoSchedule
      nodeClassRef:
        apiVersion: karpenter.k8s.aws/v1
        kind: EC2NodeClass
        name: default
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["3"]
  disruption:
    consolidationPolicy: WhenUnderutilized
    consolidateAfter: 30s
  limits:
    cpu: "1000"
    memory: 4000Gi

The consolidationPolicy: WhenUnderutilized is where Karpenter earns its cost savings: it actively merges underutilized nodes, something cluster autoscaler does more conservatively. A cluster running 8 nodes at 40% utilization can often consolidate to 4 nodes at 80%, halving your node cost.

The tradeoff: consolidation causes pod churn. Every consolidation event drains a node and reschedules its pods. On a cluster with many small workloads this happens constantly. Set consolidateAfter: 30s for batch workloads; use 30m or 1h for services where pod restarts have higher cost (cache warm-up, JVM startup time, etc.).

Namespace Resource Quotas and LimitRanges

Right-sizing individual workloads only works if teams cannot override it by setting arbitrarily large requests. Resource quotas and LimitRanges enforce fleet-wide cost governance at the namespace level.

A ResourceQuota caps the total resources a namespace can consume:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: production-quota
  namespace: team-payments
spec:
  hard:
    requests.cpu: "20"
    requests.memory: 40Gi
    limits.cpu: "40"
    limits.memory: 80Gi
    pods: "100"
    count/deployments.apps: "20"

A LimitRange sets defaults and bounds for individual pods and containers. This prevents the common failure mode where someone deploys a container with no resource requests at all (which makes it impossible to schedule or quota-track):

apiVersion: v1
kind: LimitRange
metadata:
  name: container-defaults
  namespace: team-payments
spec:
  limits:
    - type: Container
      default:
        cpu: 200m
        memory: 256Mi
      defaultRequest:
        cpu: 100m
        memory: 128Mi
      max:
        cpu: "4"
        memory: 8Gi
      min:
        cpu: 50m
        memory: 64Mi

The default values apply when a container spec omits limits. The defaultRequest applies when requests is omitted. This gives you a safe floor and ceiling without requiring every developer to understand Kubernetes resource semantics.

The operational pattern: set quotas slightly above what teams currently consume. This gives them room to grow without approval, but makes deliberate over-provisioning visible as a quota increase request. Review quotas quarterly, not annually.

Cost Attribution Within Kubernetes

Labels are the Kubernetes cost attribution primitive. Apply them consistently at the pod level and your cost tooling (Kubecost, OpenCost, cloud-native billing exports) can slice costs by team, service, environment, and feature branch.

A minimal but effective label taxonomy:

metadata:
  labels:
    app.kubernetes.io/name: payments-api
    app.kubernetes.io/component: api
    team: payments
    env: production
    cost-center: "CC-1042"

The app.kubernetes.io/* labels follow the Kubernetes recommended labels standard and are understood by most tooling. The team and cost-center labels are for your internal showback reports.

Showback (showing teams what they cost, without chargebacks) consistently changes behavior faster than policies do. When a team sees they are spending $8,000/month on staging environments that run 24/7, they fix it. When that number is invisible, nothing changes.

To surface idle resources, run a daily query against your metrics backend. The pattern: any deployment where average CPU utilization over the past 7 days is below 5% of its request is a candidate for right-sizing or teardown.

A script using the Kubernetes API and Prometheus to generate an idle workload report:

import { PrometheusDriver } from 'prometheus-query';

const prom = new PrometheusDriver({ endpoint: process.env.PROMETHEUS_URL });

async function findIdleDeployments(namespace: string): Promise<void> {
  // CPU utilization as percentage of request, averaged over 7 days
  const query = `
    sum by (pod, namespace, label_team) (
      rate(container_cpu_usage_seconds_total{
        namespace="${namespace}",
        container!="",
        container!="POD"
      }[7d])
    )
    /
    sum by (pod, namespace, label_team) (
      kube_pod_container_resource_requests{
        namespace="${namespace}",
        resource="cpu",
        container!=""
      }
    ) * 100
  `;

  const result = await prom.instantQuery(query);

  const idleThreshold = 5; // percent of request

  const idlePods = result.result
    .filter((r) => parseFloat(r.value.value as string) < idleThreshold)
    .map((r) => ({
      pod: r.metric.labels['pod'],
      namespace: r.metric.labels['namespace'],
      team: r.metric.labels['label_team'] ?? 'unknown',
      utilizationPct: parseFloat(r.value.value as string).toFixed(2),
    }));

  console.table(idlePods);
}

findIdleDeployments('production');

Run this weekly and route the output to the owning team via Slack or email. The “team” label does the attribution. The conversation shifts from “we need to cut costs” to “your payments-staging namespace has 6 idle deployments.”

Tradeoffs at Each Decision Point

Strategy	Cost Reduction	Reliability Risk	Effort
Right-size requests (VPA recommendations)	20-40%	Low, if done incrementally	Medium: requires observation period + staged rollout
Spot instances for stateless workloads	60-90% on spot nodes	Medium: spot reclamation disrupts pods	Medium: PDBs, graceful drain, fallback affinity
Cluster Autoscaler consolidation	10-25%	Low, disruption is controlled	Low: tune scale-down delay
Karpenter consolidation	20-40%	Medium: more aggressive bin-packing	Medium: NodePool config, disruption settings
Namespace quotas and LimitRanges	Indirect: prevents waste	Low	Low: one-time setup per namespace
Cost attribution and showback	Indirect: drives team behavior	None	Low: label discipline + dashboard

The highest-leverage change for most clusters is spot instances for stateless workloads, because the cost reduction is immediate and the engineering investment is bounded. Right-sizing is the second priority and has no reliability risk if done gradually.

Production Pitfalls

Spot interruptions during peak traffic: the cloud provider does not know your traffic patterns. Spot reclamations happen when demand for spot capacity is high in a region, which can correlate with everyone running more workloads simultaneously. Build capacity headroom into your PDB settings so a reclamation during a traffic spike does not reduce you below minimum viable replicas.

VPA and HPA conflict: if you run both VPA and HPA on the same deployment with CPU-based scaling, they interfere. HPA scales replica count based on CPU utilization; VPA changes the CPU requests those replicas have. This creates feedback loops. The safe configuration: use VPA in Off mode (recommendations only) when HPA is managing a deployment. Apply VPA recommendations manually, then redeploy. Never run VPA Auto mode alongside CPU-based HPA.

Scale-down delay misconfiguration: cluster autoscaler’s --scale-down-delay-after-add defaults to 10 minutes. In clusters with spiky traffic, nodes provisioned for a spike get deprovisioned 10 minutes later, then reprovisioned for the next spike. You pay node startup latency twice and your scaling lags behind demand. For services with daily traffic patterns, increase this to 30-60 minutes during peak hours.

Missing PDBs on critical services: the cluster autoscaler and Karpenter both respect PDBs during voluntary node drains. A service without a PDB can have all its pods evicted simultaneously during a consolidation event. Audit your critical services for PDB coverage before enabling aggressive consolidation.

Overquota errors blocking deployments: when a namespace’s ResourceQuota is exhausted, new pod scheduling fails silently from the developer’s perspective. The deployment succeeds, but pods stay in Pending state with a quota error in events. Add monitoring on kube_resourcequota metrics to alert when any namespace is above 80% of its quota limit.

Closing Observation

Kubernetes cost optimization is not a one-time project. The cluster state drifts continuously: new services get deployed with default (over-provisioned) requests, traffic patterns shift, teams add feature branch deployments that persist after the feature ships. The teams with predictable cloud bills treat it as ongoing housekeeping, not a periodic firefight. The tooling (VPA recommendations, Karpenter consolidation, showback dashboards) does most of the detection work. The engineering investment is in the guardrails: quotas, LimitRanges, PDBs, and label discipline. Put those in place once and you have a system that resists drift rather than one that requires constant manual intervention to stay lean.