The Hidden Cost of the Legacy Cluster Autoscaler

To understand why your EKS cluster is bleeding money, you have to understand the foundational mechanics of how the standard Cluster Autoscaler operates—and where it falls short in modern, dynamic cloud environments. 

CAS watches the Kubernetes API server for pods that cannot be scheduled due to resource constraints. When it sees pending pods, it calculates if adding a node will help, and then adjusts the DesiredCapacity of your AWS Auto Scaling Groups. 

This introduces three critical bottlenecks that directly impact a scaling startup's bottom line:

1. The Rigid Node Group Problem

CAS relies entirely on Node Groups, which are backed by AWS ASGs. These groups are almost always homogeneous, meaning they are constructed of instances with identical CPU and memory ratios (e.g., all m5.xlarge). 

Consider this scenario: A pod requires 1 vCPU and 2GB of RAM to schedule. However, your ASG is configured to spin up m5.2xlarge instances (8 vCPUs, 32GB RAM). Because CAS is blind to the actual AWS compute catalog and only knows how to trigger the ASG, it spins up an instance with massive compute overhead just to house a tiny pod. The rest of those resources go unutilized, yet you pay the full hourly rate. To combat this, DevOps teams are forced to create dozens of highly specific Node Groups to handle different workload profiles, creating a fragmented operational nightmare that is impossible to maintain efficiently.

2. Inefficient Bin-Packing (The Kubernetes Tetris)

Kubernetes scheduling is essentially a massive game of Tetris. Because CAS is chained to the limitations of ASGs, it struggles to play this game well. It doesn't analyze the exact optimal instance type across the entire AWS catalog for your specific, real-time combination of pending pods. Instead, it blindly triggers the ASG to increment its count.

The result? A cluster full of EC2 instances that sit 30-40% idle at any given time, creating a massive compute buffer paid for by your startup.

3. Sluggish Scheduling and The Spin-Up Lag

Speed is a feature. When a pod goes pending under CAS, a painfully slow chain of events kicks off:

1. CAS detects the pending pod.

2. CAS talks to the AWS API to update the ASG.

3. The ASG requests an instance from EC2.

4. EC2 provisions the instance and it boots up.

5. The instance joins the cluster via the `kubelet`.

6. The node finally becomes `Ready` and the pod is scheduled.

This lifecycle frequently takes 2 to 4 minutes. In a bursty, high-traffic environment, that latency translates directly to dropped requests, degraded user experience, and a fragile CI/CD pipeline.

Enter Karpenter: Just-in-Time Provisioning for the Modern Cloud

Developed initially by AWS and now an incubated project in the Cloud Native Computing Foundation (CNCF), Karpenter represents a fundamental paradigm shift in Kubernetes autoscaling. It bypasses Auto Scaling Groups entirely.

Instead of managing rigid Node Groups, Karpenter watches for unschedulable pods, evaluates their exact resource requests (CPU, memory, GPUs, volume topology, and node selectors), and makes a direct call to the Amazon EC2 Fleet API. 

Why Karpenter is a Game Changer for Startup FinOps

Group-less, Dynamic Architecture: Karpenter is not constrained by predefined instance types. If you have a batch of pending pods that perfectly fit into an r6g.xlarge (ARM Graviton), Karpenter will provision exactly that instance. If the next batch requires a compute-optimized c6i.large, it spins that up instead. It evaluates the entire AWS catalog in real-time.

Sub-Minute Scheduling: By cutting out the ASG middleman and directly leveraging the EC2 Fleet API, Karpenter routinely provisions and registers new EC2 nodes in under 40 seconds. This transforms how your application handles traffic spikes.

Continuous Consolidation: This is where the massive cost savings happen. Karpenter actively monitors cluster utilization on an ongoing basis. If it notices that workloads spread across three mostly empty instances could comfortably fit onto one smaller, cheaper instance, it will gracefully cordon, drain, and terminate the bloated nodes, replacing them with a single right-sized instance. This active, automated bin-packing ensures you are never paying for idle compute.

The Spot Instance Goldmine (Safely Managed)

Spot Instances offer up to 90% off On-Demand EC2 prices, making them the holy grail of cloud cost reduction. But they come with a catch: AWS can reclaim them with a 2-minute warning. 

Managing Spot interruptions with traditional ASGs and the AWS Node Termination Handler is complex, fragile, and often leads to downtime. Karpenter simplifies this natively and elegantly. 

By integrating directly with Amazon SQS and Amazon EventBridge, Karpenter listens for Spot Interruption warnings at the AWS infrastructure level. When a termination notice is received, Karpenter instantly cordons the dying node and begins provisioning a before the original node is taken away, ensuring zero downtime for your services.

Mixing Spot and On-Demand in Production

With Karpenter’s NodePool API, you can easily define highly granular rules for which workloads get Spot instances and which get On-Demand instances. 

For example, you can default your entire cluster's worker nodes to use Spot instances, while using simple node selectors or labels to ensure your stateful databases, Redis caches, or critical ingress controllers always land on highly available On-Demand instances.

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: default-spot
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h # Proactively recycle nodes every 30 days for security patching

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: default-spot
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h # Proactively recycle nodes every 30 days for security patching

This single configuration allows Karpenter to choose from hundreds of possible instance types (seamlessly mixing Intel and ARM-based Graviton architectures) across the C, M, and R compute families. It will automatically pick the cheapest available Spot instance that precisely fits the pending pods in real-time.

---

A Zero-Downtime Migration Blueprint

Migrating from the legacy Cluster Autoscaler to Karpenter doesn't require an agonizing maintenance window, a massive migration team, or a complete cluster rebuild. Because the two controllers operate differently, you can run them side-by-side during the transition. 

Here is the blueprint we use at Betta to execute this seamlessly for our startup clients:

Phase 1: Prerequisites and IAM

1. Infrastructure Tagging: Ensure your EKS subnets and Security Groups are appropriately tagged so Karpenter knows exactly where and how to launch nodes (e.g., `karpenter.sh/discovery: your-cluster-name`).

2. IRSA (IAM Roles for Service Accounts): Security is paramount. Create a strictly scoped IAM role for the Karpenter controller with permissions to manage EC2 instances, and associate it with a Kubernetes Service Account via OIDC.

3. SQS Queue Integration: Set up the SQS queue and EventBridge rules for native Spot interruption handling to guarantee application stability.

Phase 2: Deployment and Configuration

1. Deploy the Controller: Install Karpenter via its official Helm chart, ensuring you configure it to use the IAM role created in Phase 1.

2. Define EC2NodeClass and NodePools: Deploy your `EC2NodeClass` (which defines AWS-specific configurations like AMI families, subnets, and security groups) and your `NodePools` (which define Kubernetes-specific constraints like capacity types and instance families).

Phase 3: The Cordon and Drain (The Cutover)

At this stage, both CAS and Karpenter are running simultaneously, but managing different resources. To shift workloads over safely:

1. Taint the Legacy Node Groups: Apply a `NoSchedule` taint to your existing CAS-managed ASG nodes. This prevents any new pods from landing on the old, expensive infrastructure.

2. Trigger Scale-Up: As you manually delete pods on the old nodes (or use an automated draining script), those pods will go into a `Pending` state.

3. Karpenter Takes Over: Because the old nodes are tainted, CAS cannot use them to house the pending pods. Karpenter, however, immediately sees the pending pods, spins up hyper-optimized, right-sized EC2 instances via the Fleet API, and schedules them in seconds.

4. Decommission CAS: Once all legacy nodes are drained and empty, scale your old ASGs to `0`, delete the legacy Node Groups entirely, and uninstall the Cluster Autoscaler deployment from your cluster.

Phase 4: Observability and Tuning

Post-migration, it is critical to monitor Karpenter's decision-making. By scraping Karpenter's native Prometheus metrics (karpenter_nodes_created, karpenter_nodes_terminated, karpenter_pod_startup_time_seconds), your team can visualize the exact cost savings and scheduling speed improvements in Grafana, proving the ROI of the migration to your executive team.

---

The Real-World Impact: How Betta Transforms FinOps

At Betta, we specialize in solving these exact infrastructure bottlenecks for scaling B2B startups. Recently, a Series B SaaS client came to us with a rapidly ballooning AWS bill. Their engineering team was incredibly talented, but they were too busy shipping product features to untangle their sprawling, static EKS Node Groups. 

By implementing Karpenter as a fractional SRE team, we achieved the following within 30 days:

Reduced Total Node Count by 35%: Karpenter's active consolidation eliminated massive amounts of idle CPU buffer that the client had been paying for over the last year.

Shifted 80% of Non-Prod to Spot: Leveraging Karpenter's native interruption handling allowed safe, aggressive use of Spot instances for CI/CD pipelines and staging environments without breaking developer workflows.

46% Reduction in EKS Compute Costs: The combined effect of safely leveraging Spot instances and continuous right-sizing resulted in nearly cutting their total EKS cluster bill in half.

Improved Deployment Velocity: Pod scheduling times dropped from ~3.5 minutes to under 40 seconds, drastically speeding up developer feedback loops and horizontal scaling during traffic spikes.

The Fractional Advantage for Scaling Startups

Platform engineering and deep FinOps optimization are rarely one-time projects, but hiring a full-time, $200k+ FinOps engineer or specialized Kubernetes architect to manage this single transition is often overkill for a lean startup. 

This is the power of the fractional model. By bringing in specialized experts to architect, execute, and document high-leverage migrations like Karpenter, you preserve your startup's runway, eliminate technical debt, and most importantly, keep your full-time engineering team focused squarely on your core product and revenue-generating features.

Stop letting rigid, outdated infrastructure dictate your cloud bill. The tools to build elastic, highly cost-efficient Kubernetes clusters exist today—you just need the right expertise to implement them flawlessly.