In Kubernetes, scaling your cluster to meet the needs of your applications is crucial for ensuring high availability and performance. One way to optimize the costs of your cluster is by using KEDA (Kubernetes-based Event Driven Autoscaler) and Karpenter

What is KEDA?

KEDA allows you to scale the number of pods in your cluster based on the number of events that need to be processed. This can help you save money by only running the number of pods necessary to handle your workload.

What is Karpenter ? How can you optimize costs with auto scaling nodes?

Karpenter automatically launches just the right compute resources to handle your cluster’s applications. It is designed to let you take full advantage of the cloud with fast and simple compute provisioning for Kubernetes clusters. It Also helps you to reduce costs by using EC2 Spot Instances. Spot Instances allow you to bid on spare Amazon EC2 computing capacity and can save you up to 90% compared to On-Demand prices. By using Karpenter to scale your cluster, you can take advantage of these cost savings while still maintaining the necessary capacity for your applications.

To use Karpenter, you will need to create a Custom Provisioner resource in your cluster. This resource specifies the requirements for the nodes that Karpenter should provision, such as the instance type and availability zone.

To get started with KEDA and Karpenter in an EKS cluster, you need to install the KEDA and Karpenter controllers and create a Provisioner resource

Keda: https://keda.sh/docs/2.9/deploy/

Karpenter: https://karpenter.sh/

Here is an example Provisioner resource that uses EC2 Spot Instances:

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: spot-fleet
spec:
  kubeletConfiguration:
    containerRuntime: dockerd
  taints:
    - key: spot-fleet.io/specialfe-taint
      value: "true"
      effect: NoSchedule
  requirements:
    - key: topology.kubernetes.io/zone
      voperator: In
      values: [eu-west-1a, eu-west-1b, eu-west-1c] # Zones
    - key: "karpenter.sh/capacity-type" # If not included, the webhook for the AWS cloud provider will default to on-demand
      operator: In
      values: ["spot", "on-demand"]
    - key: "karpenter.k8s.aws/instance-category"
      operator: In
      values: ["c", "m", "r"]
    - key: "karpenter.k8s.aws/instance-cpu"
      operator: In
      values: ["4", "8", "16", "32"]
  provider:
    subnetSelector:
       Name: "dev-vpc-*"
    securityGroupSelector:
      Name: 'eks-cluster-sg-spot-fleet-123444'
  ttlSecondsAfterEmpty: 30
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: spot-fleet
spec:
  requirements:
    - key: topology.kubernetes.io/zone
      operator: In
      values: [eu-west-1a, eu-west-1b, eu-west-1c]
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
    - key: "karpenter.k8s.aws/instance-category"
      operator: In
      values: ["c", "m", "r"]
    - key: "karpenter.k8s.aws/instance-cpu"
      operator: In
      values: ["4", "8", "16", "32"]
  provider:
    subnetSelector:
      karpenter.sh/discovery: ${CLUSTER_NAME}
    securityGroupSelector:
      Name: 'eks-cluster-sg-spot-fleet-2345'
  ttlSecondsUntilExpired: 2592000
  ttlSecondsAfterEmpty: 30 

This Provisioner resource specifies that Karpenter should provision nodes in the specified availability zones that are using EC2 Spot Instances and have either 4, 8, 16, or 32 CPUs.

Sample Nginx Deployment with Karpenter Node :-

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      nodeSelector:
        karpenter.k8s.aws/instance-category: c
        karpenter.k8s.aws/instance-cpu: '4'
        karpenter.sh/capacity-type: spot
      containers:
      - name: my-app
        image: my-app:latest
        ports:
        - containerPort: 80

By using node selectors and tolerations, you can deploy pods to specific nodes provisioned by Karpenter. These nodes can be either on-demand or spot instances. The node selectors and tolerations are specified as labels, which allows you to fine-tune the distribution of your pods across your cluster for better resource utilisation and cost optimisation.