amazon-eks-ami

The nodeadm API types (under the node.eks.aws API group) follow Kubernetes-style versioning with three stability levels:

• Alpha (e.g., v1alpha2): Support may be removed at any time. Incompatible changes may occur without migration instructions.
• Beta (e.g., v3beta4): Support remains for at least one release following deprecation. Incompatible changes in subsequent beta or stable versions will include migration instructions.
• Stable (e.g., v5): Support aligns with the support lifecycle of the major Amazon Linux version used.

On EKS AMIs, you can override settings provided in EC2 user data by placing NodeConfig objects (as .yaml, .yml, or .json files) into /etc/eks/nodeadm.d/.

Drop-in files are evaluated after user data, so they take precedence. A drop-in file only needs to contain the specific fields you wish to override. Note that at least one source (typically user data) must still provide the required cluster fields.

### Example drop-in file at `/etc/eks/nodeadm.d/10-rlimits.yaml`
```yaml
---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  containerd:
    baseRuntimeSpec:
      process:
        rlimits:
          - type: RLIMIT_NOFILE
            soft: 65536
            hard: 65536

When using the IMDS configuration source (--config-source=imds://user-data), nodeadm merges all configuration objects found in a MIME multi-part document. Objects are merged in the order they appear; values in later objects take precedence over earlier ones.

MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="BOUNDARY"

--BOUNDARY
Content-Type: application/node.eks.aws

---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  cluster:
    name: my-cluster
    ...

--BOUNDARY
Content-Type: application/node.eks.aws

---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  kubelet:
    config:
      shutdownGracePeriod: 30s

--BOUNDARY--

The maximum number of pods schedulable on a node is determined by the number of ENIs available for an instance type. To optimize performance, default values for known instance types are cached in a resource file. If an instance type is missing from this cache, the system falls back to calling the ec2:DescribeInstanceTypes API at runtime.

You can regenerate this resource file by running the generate-instance-info target within the nodeadm directory.

cd nodeadm
make generate-instance-info

To build an AMI using specific Kubernetes versions and build dates, you must first locate the correct paths in the amazon-eks S3 bucket. You can use the AWS CLI to list versions, build dates, platforms, architectures, and binaries.

Once you have identified the values, pass them to the make k8s command.

# Example build command
make k8s \
  kubernetes_version=1.23.9 \
  kubernetes_build_date=2022-07-27 \
  arch=x86_64

G7 instances require NVIDIA driver version 595 or later. Because the default EKS-optimized accelerated AMI ships with driver version 580, you must build a custom AMI to support G7 instances.

Warning: Driver version 595 is not compatible with P3, P3dn, and G6f instance types. Do not use this AMI for those instances.

Prerequisites

• Packer installed
• AWS credentials configured with permissions to create AMIs (EC2, S3 read access)
• Clone the repository:

git clone https://github.com/awslabs/amazon-eks-ami.git
cd amazon-eks-ami

Before building an AMI, ensure you have the following requirements met:

1. Packer: Version 1.8.0 or later must be installed on your local system.
2. AWS Credentials: You must have AWS account credentials configured so that Packer can perform AWS API operations on your behalf.

To override default container RLIMITs, use the containerd.baseRuntimeSpec.process.rlimits path in your NodeConfig. This can be applied via user data or a drop-in file in /etc/eks/nodeadm.d/.

---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  cluster: ...
  containerd:
    baseRuntimeSpec:
      process:
        rlimits:
          - type: RLIMIT_NOFILE
            soft: 1024
            hard: 1024

To reduce the latency of nodes reaching a Ready state, you can cache specific container images (like kube-proxy and amazon-k8s-cni) during the AMI build process. This is done by setting the cache_container_images variable to true when running the make command.

cache_container_images=true make 1.23

You can provide additional containerd configuration using an inline TOML document within the NodeConfig. These values will overwrite any defaults set by nodeadm.

---
apiVersion: node.eks.aws/v1alpha1
kind: NodeConfig
spec:
  cluster: ...
  containerd:
    config: |
      [plugins."io.containerd.grpc.v1.cri".containerd]
      discard_unpacked_layers = false