eksctl

repository·main·Indexed 26 days ago

https://github.com/eksctl-io/eksctl

The official CLI tool for Amazon EKS, designed to simplify the creation and management of Kubernetes clusters on AWS using CloudFormation. It includes support for an MCP server for Amazon Q Chat integration, node bootstrapping for Amazon Linux 2, Amazon Linux 2023 (via nodeadm), and Ubuntu, and integrates AWS GoFormation for programmatically building and parsing CloudFormation or SAM templates using Go structs.

Tokens
60.3K
Snippets
171
Records
302
Agent score
89%

What's inside eksctl

  1. Understand the deprecated Node Bootstrap design

    main

    DEPRECATION NOTICE

    This node bootstrap design is DEPRECATED as of version 0.47.0. For current implementation details, refer to pkg/nodebootstrap/README.md in the repository.

    Design Overview

    The legacy design aimed for a stateless, immutable, and deterministic bootstrap process. It relied on passing configuration files via Cloud-init userData to the node.

    Key characteristics:

    • Configuration files are written to the /etc/eksctl directory.
    • Uses the EC2 Metadata Service (IMDSv2) to retrieve instance ID, type, and IP address.
    • Avoids heavy dependencies like jq or aws-cli during the bootstrap stage to minimize security surface area and runtime requirements.
  2. Understand Nodebootstrap implementation for unmanaged nodes

    main

    For unmanaged nodes, eksctl uses a Bootstrapper interface to provide UserData. As of version 0.45.0, eksctl defers to the native bootstrap script built into the AMI (located at /etc/eks/bootstrap.sh) rather than using its own internal logic. The UserData provided by eksctl acts as a wrapper script that applies custom configurations and then delegates execution to the official /etc/eks/bootstrap.sh script.

    type Bootstrapper interface {
      UserData() (string, error)
    }
  3. Understand default VPC networking behavior

    main

    When running eksctl create cluster, a dedicated VPC is created by default to prevent interference with existing resources.

    Default VPC Specifications:

    • CIDR Block: 192.168.0.0/16.
    • Subnet Layout: Divided into 8 /19 subnets (3 private, 3 public, and 2 reserved).
      • Note: In us-east-1, eksctl only creates 2 public and 2 private subnets by default.
    • Nodegroup Placement: The initial nodegroup is created in public subnets.
    • SSH Access: Disabled by default unless the --allow-ssh flag is specified.
    • Inbound Traffic: Nodegroups allow inbound traffic from the control plane security group on ports 1025 - 65535.
  4. Configure Zone-aware Auto Scaling

    main

    The cluster-autoscaler assumes all nodes in a single nodegroup are equivalent. If your workloads have zone-specific requirements (such as EBS volumes/PVCs tied to a specific Availability Zone), a single multi-AZ nodegroup may cause pods to fail if they are scheduled in the wrong AZ.

    When to use separate nodegroups per AZ

    You should create separate, single-AZ nodegroups if your workloads have:

    • Zone-specific storage requirements (e.g., EBS volumes).
    • podAffinity requirements with topology other than host.
    • nodeAffinity requirements on zone labels.
    • nodeSelector requirements on zone labels.

    Implementation Example

    Instead of one nodegroup spanning multiple zones, define one nodegroup per zone:

    nodeGroups:
      - name: ng1-public-2a
        instanceType: m5.xlarge
        availabilityZones: ["eu-west-2a"]
      - name: ng1-public-2b
        instanceType: m5.xlarge
        availabilityZones: ["eu-west-2b"]
    nodeGroups:
      - name: ng1-public-2a
        instanceType: m5.xlarge
        availabilityZones: ["eu-west-2a"]
      - name: ng1-public-2b
        instanceType: m5.xlarge
        availabilityZones: ["eu-west-2b"]
  5. List, Update, and Delete EKS Addons

    main

    Manage existing addons using the following commands:

    • List addons: View enabled addons in a cluster.
    • Update addons: Update versions or IAM policies. Note that resolveConflicts is also used here (none, overwrite, or preserve).
    • Delete addons: Removes the addon and its associated IAM roles.
    # List addons
    eksctl get addons --cluster <cluster-name>
    eksctl get addons -f config.yaml
    
    # Update addons
    eksctl update addon -f config.yaml
    eksctl update addon --name vpc-cni --version 1.8.0 --service-account-role-arn <new-role>
    
    # Delete addons
    eksctl delete addon --cluster <cluster-name> --name <addon-name>
  6. Unmarshal CloudFormation YAML/JSON into Go structs

    main

    Use goformation.Open(filename) to parse an existing JSON or YAML CloudFormation/SAM template into a Go template object. Once opened, you can interact with the resources using typed methods:

    • Use GetAllServerlessFunctionResources() to retrieve all AWS::Serverless::Function resources.
    • Use GetServerlessFunctionWithName(name) to find a specific function by its logical ID.
    package main
    
    import (
    	"log"
    
    	"github.com/awslabs/goformation/v4"
    )
    
    func main() {
    	// Open a template from file (can be JSON or YAML)
    	template, err := goformation.Open("template.yaml")
    	if err != nil {
    		log.Fatalf("There was an error processing the template: %s", err)
    	}
    
    	// You can extract all resources of a certain type
    	// Each AWS CloudFormation resource is a strongly typed struct
    	functions := template.GetAllServerlessFunctionResources()
    	for name, function := range functions {
    
    		// E.g. Found a AWS::Serverless::Function named GetHelloWorld (runtime: nodejs6.10)
    		log.Printf("Found a %s named %s (runtime: %s)\n", function.AWSCloudFormationType(), name, function.Runtime)
    
    	}
    
    	// You can also search for specific resources by their logicalId
    	search := "GetHelloWorld"
    	function, err := template.GetServerlessFunctionWithName(search)
    	if err != nil {
    		log.Fatalf("Function not found")
    	}
    
    	// E.g. Found a AWS::Serverless::Function named GetHelloWorld (runtime: nodejs6.10)
    	log.Printf("Found a %s named %s (runtime: %s)\n", function.AWSCloudFormationType(), search, function.Runtime)
    }
  7. Join the eksctl community on Slack

    main

    If you need technical support, have questions, or want to get in touch with the core team, you can join the community via the Kubernetes Slack workspace. Use the #eksctl channel for discussions related to this project.

    https://slack.k8s.io/messages/eksctl/
  8. Enable access for Amazon EMR Containers

    main

    To allow Amazon EMR to perform operations on the Kubernetes API, you must grant its Service Linked Role (SLR) the required RBAC permissions.

    Use the eksctl create iamidentitymapping command to create the necessary RBAC resources for EMR and update the aws-auth ConfigMap to bind the EMR role with the SLR.

    $ eksctl create iamidentitymapping --cluster dev --service-name emr-containers --namespace default
  9. Disable EKS Auto Mode

    main

    To disable Auto Mode on an existing cluster, set autoModeConfig.enabled: false in your configuration file and run the update command.

    # cluster.yaml
    apiVersion: eksctl.io/v1alpha5
    kind: ClusterConfig
    metadata:
        name: auto-mode-cluster
        region: us-west-2
    
    autoModeConfig:
        enabled: false
    
    $ eksctl update auto-mode-config -f cluster.yaml
  10. Configure custom Windows AMIs

    main

    Only self-managed Windows nodegroups can specify a custom AMI. You must set a valid Windows amiFamily and provide an overrideBootstrapCommand using PowerShell.

    The following PowerShell variables are available within your bootstrap script:

    • $EKSBootstrapScriptFile
    • $EKSClusterName
    • $APIServerEndpoint
    • $Base64ClusterCA
    • $ServiceCIDR
    • $KubeletExtraArgs
    • $KubeletExtraArgsMap (a hashtable containing arguments like @{ 'node-labels' = ''; 'register-with-taints' = ''; 'max-pods' = '10'})
    • $DNSClusterIP
    • $ContainerRuntime
    nodeGroups:
      - name: custom-windows
        amiFamily: WindowsServer2022FullContainer
        ami: ami-01579b74557facaf7
        overrideBootstrapCommand: |
          & $EKSBootstrapScriptFile -EKSClusterName "$EKSClusterName" -APIServerEndpoint "$APIServerEndpoint" -Base64ClusterCA "$Base64ClusterCA" -ContainerRuntime "containerd" -KubeletExtraArgs "$KubeletExtraArgs" 3>&1 4>&1 5>&1 6>&1
  11. Configure Cross Account Pod Identity Associations

    main

    eksctl supports EKS Pod Identity cross-account access, allowing pods in your EKS cluster to access AWS resources in a different AWS account.

    To implement this:

    1. Configure IAM Roles and Policies in both the source account (where the cluster resides) and the target account (where the resources reside) to allow access.
    2. Use an eksctl cluster configuration file to define the podIdentityAssociations under the iam section.

    Note: The name of the cluster and the serviceAccountName must match the trust relationship defined in the target account's IAM policy.

    apiVersion: eksctl.io/v1alpha5
    kind: ClusterConfig
    metadata:
      name: my-cluster
      region: us-west-2
      version: "1.32"
    
    addons:
      - name: vpc-cni
      - name: coredns
      - name: kube-proxy
      - name: eks-pod-identity-agent
    
    iam:
      podIdentityAssociations:
      - namespace: default
        serviceAccountName: demo-app-sa
        createServiceAccount: true
        # The source role in the same account as the cluster
        roleARN: arn:aws:iam::1111111111:role/account-a-role
        # The target role in a different account
        targetRoleARN: arn:aws:iam::2222222222:role/account-b-role
        # Optional: Disable session tags
        disableSessionTags: false
    
    managedNodeGroups:
      - name: my-cluster
        instanceType: m6a.large
        privateNetworking: true
        minSize: 2
        desiredCapacity: 2
        maxSize: 3
  12. Upgrade a simple cluster with a single unmanaged nodegroup

    main

    If your cluster has only one initial nodegroup, follow these steps to upgrade it by replacing the old one with a new one:

    1. Identify the old nodegroup name:
      eksctl get nodegroups --cluster=<clusterName> --region=<region>
    2. Create the new nodegroup:
      eksctl create nodegroup --cluster=<clusterName> --region=<region> --name=<newNodeGroupName> --managed=false
    3. Delete the old nodegroup:
      eksctl delete nodegroup --cluster=<clusterName> --region=<region> --name=<oldNodeGroupName>

    Important: eksctl delete nodegroup will attempt to drain all pods from the nodegroup before deleting instances. If Pod Disruption Budget (PDB) policies prevent eviction, use the --disable-eviction flag to bypass PDB checks and force deletion.

    eksctl get nodegroups --cluster=<clusterName> --region=<region>
    
    eksctl create nodegroup --cluster=<clusterName> --region=<region> --name=<newNodeGroupName> --managed=false
    
    eksctl delete nodegroup --cluster=<clusterName> --region=<region> --name=<oldNodeGroupName>