Amazon EFS CSI Driver

repository·master·Indexed 21 days ago

https://github.com/kubernetes-sigs/aws-efs-csi-driver

The Amazon EFS CSI Driver allows Kubernetes clusters on AWS to mount Amazon EFS and Amazon S3 file systems as persistent volumes. It supports dynamic and static provisioning, cross-account mounting, and encryption in transit. Starting from version 3.0.0, the driver supports Amazon S3 file systems. Key features include the use of EFS Access Points for independent data store ownership and multiarch image support on ECR.

Tokens
33.2K
Snippets
72
Records
136
Agent score
72%

What's inside aws-efs-csi-driver

  1. Overview of Amazon EFS CSI Driver

    master

    The Amazon EFS CSI Driver enables Kubernetes clusters running on AWS to mount both Amazon EFS and Amazon S3 file systems (starting from driver version 3.0.0 or above) as persistent volumes.

    Key capabilities include:

    • Dynamic Provisioning: Automatically creates an EFS or S3 access point when a Persistent Volume Claim (PVC) is created.
    • Static Provisioning: Allows mounting existing, manually created EFS or S3 file systems.
    • Encryption: Supports encryption in transit by default (in the master branch).
    • Cross-account mounting: Supports mounting EFS file systems from different AWS accounts.
    • Multiarch support: The driver image is available as a multiarch image on ECR.

    Important Notes:

    • Capacity Placeholders: Since EFS and S3 are fully elastic, the storage capacity specified in PersistentVolumes (PV) and PersistentVolumeClaims (PVC) is a required Kubernetes placeholder and does not affect actual scaling or billing.
    • Avoid Node-level Installation: Do not install amazon-efs-utils directly on EKS worker nodes. The driver packages efs-utils within its own containers and manages the mount process. Installing it at the node level can cause unexpected behavior.
  2. Installation considerations for Amazon EFS CSI Driver

    master

    Before installing the Amazon EFS CSI Driver, be aware of the following compatibility and feature constraints:

    • OS Compatibility: Not compatible with Windows-based container images.
    • AWS Fargate: Dynamic persistent volume provisioning is not supported on Fargate nodes; use static provisioning instead. (Note: Pods on Fargate automatically mount EFS/S3 without manual driver installation).
    • Architecture: Arm64 (Amazon EC2 Graviton) is supported starting from version 1.3.2.
    • FIPS: FIPS support for mounting file systems requires version 1.4.2 or later.
    • Amazon S3 Files: Supported starting from version 3.0.0.
    • Dynamic Provisioning: Requires driver version 1.2 or later.
    • Resource Quotas: Be mindful of Amazon EFS limits, such as the 10,000 access points quota per file system.
  3. How AZ availability affects EFS mount connectivity

    master

    The impact of an Availability Zone (AZ) outage depends on your provisioning mode:

    1. DNS mode (crossaccount=true in the secret): Nodes resolve mount targets via DNS at mount time. Pods in healthy AZs continue working; pods in the failed AZ lose connectivity until rescheduled.
    2. Default per-node AZ selection (no az or crossaccount specified): The controller resolves all mount targets at provisioning. Nodes select their own AZ's mount target. If a node's AZ has no mount target, it falls back to an available one (logged as a warning). Pods in healthy AZs continue working.
    3. Pinned AZ (StorageClass az=<zone>): Every pod is forced to use the mount target in the specified zone. If that AZ goes down, all pods using that StorageClass lose connectivity.

    Note: If using an older driver version that baked a single IP into PVs, upgrade to v3.1.0 or newer to enable the default per-node AZ selection feature.

  4. How the driver selects EFS mount targets

    master

    The driver determines how to mount an EFS file system based on the combination of the provisioner secret and the StorageClass configuration. This selection logic dictates whether the driver uses DNS resolution, a specific IP, or automatic AZ-aware selection.

    TriggerPV volumeAttributes writtenMount-time behavior
    Secret has crossaccount=truecrossaccount: "true"Each node uses efs-utils DNS resolution to mount a target in its own AZ. No IP is baked into the PV.
    StorageClass has az=<zone> (and crossaccount unset/false)mounttargetip: <ip>Every node uses the single mount target IP baked into the PV. Falls back to a random target if the specified AZ is unavailable.
    Neither set (default)(internal AZ→IP mapping)Each node selects the mount target in its own AZ; falls back to any available target if the local AZ is absent.

    Note: For static provisioning, you must explicitly set crossaccount: "true" or mounttargetip: <ip> in the PV's volumeAttributes to control this behavior.

  5. Required IAM policies for EFS CSI Driver roles

    master

    The EFS CSI driver requires two distinct service accounts with specific IAM permissions to function correctly:

    1. Controller Role (efs-csi-controller-sa)

    Used by the driver controller. Requires:

    • AmazonEFSCSIDriverPolicy
    • AmazonS3FilesCSIDriverPolicy
    • AmazonS3FilesClientFullAccess

    2. Node Role (efs-csi-node-sa)

    Used by the node daemonset. Requires:

    • AmazonS3FilesClientFullAccess: Grants permissions to mount and write to S3 file systems (s3files:ClientMount, s3files:ClientWrite, s3files:ClientRootAccess).
    • AmazonS3ReadOnlyAccess: Enables direct S3 read access for high-throughput object streaming.
    • AmazonElasticFileSystemsUtils: Enables publishing efs-utils logs to Amazon CloudWatch for troubleshooting.
  6. Use Dynamic Filesystem ID Resolution

    master

    The EFS CSI driver can automatically resolve EFS filesystem IDs from Kubernetes ConfigMaps or Secrets. This is useful when external tools provision EFS filesystems and store the resulting IDs in Kubernetes resources.

    To use this feature, you must provide a reference in the StorageClass parameters using the format: namespace/name/key.

    Parameter Constraints: You must specify exactly one of the following in your StorageClass parameters:

    • fileSystemId: The raw filesystem ID string.
    • fileSystemIdConfigRef: A reference to a ConfigMap in the format namespace/name/key.
    • fileSystemIdSecretRef: A reference to a Secret in the format namespace/name/key.
    # Example: Using a ConfigMap to provide the filesystem ID
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: efs-config
      namespace: kube-system
    data:
      fileSystemId: fs-02604354c13d0316d
    ---
    kind: StorageClass
    apiVersion: storage.k8s.io/v1
    metadata:
      name: efs-sc
    provisioner: efs.csi.aws.com
    parameters:
      provisioningMode: efs-ap
      fileSystemIdConfigRef: "kube-system/efs-config/fileSystemId"
      directoryPerms: "700"
  7. Prevent race conditions using node startup taints

    master

    To prevent pods from running on a node before the EFS CSI Driver is fully ready (especially during node startup), you can use the driver's automatic taint removal feature. This feature is enabled by default in driver version v1.7.2+ and Helm chart v2.5.2+.

    How to use it:

    1. Taint your nodes when they join the cluster or on startup.
    2. Use the specific taint key: efs.csi.aws.com/agent-not-ready:NoExecute (though any effect works, NoExecute is recommended).

    This ensures that pods requiring EFS will not be scheduled on a node until the driver agent is ready to handle them.

    efs.csi.aws.com/agent-not-ready:NoExecute
  8. Provisioning modes in Amazon EFS CSI Driver

    master

    The driver supports two primary methods for managing storage:

    Static Provisioning

    Use this when you have already created an Amazon EFS or Amazon S3 file system manually. You then define a PersistentVolume (PV) in Kubernetes to point to that existing resource.

    Note for S3 Static Provisioning: When using static provisioning with an Amazon S3 file system, the volumeHandle in your PersistentVolume must include the s3files: prefix.

    Example format: s3files:fs-01234567890abcdef0

    Dynamic Provisioning

    Use this to automate storage management. When a user creates a Persistent Volume Claim (PVC), the driver automatically requests Amazon EFS or Amazon S3 to create an access point, which is then used to mount the volume.

  9. How Amazon EFS access points work

    master

    The Amazon EFS CSI driver supports Amazon EFS access points. Access points act as application-specific entry points into an EFS file system. They are useful for sharing a single file system between multiple Pods by providing:

    1. Identity Enforcement: They can enforce a specific user identity (POSIX user/group) for all file system requests made through that access point.
    2. Root Directory Enforcement: They can enforce a specific root directory for each Pod, providing isolation and simplifying path management.

    For detailed implementation details, refer to the Amazon EFS access points documentation.

  10. Dynamic Provisioning workflow

    master

    Dynamic provisioning automates the creation of EFS Access Points when a user requests storage via a StorageClass and PVC. The driver manages the entire lifecycle from creation to deletion.

    Workflow Phases:

    1. Provisioning Phase

    • Trigger: User creates a StorageClass and a PVC.
    • Action: The csi-provisioner detects the PVC and calls the CreateVolume RPC on the Controller.
    • EFS Creation: The Controller parses parameters (like UID/GID) from the StorageClass, calls the EFS CreateAccessPoint API, and generates a VolumeId in the format fileSystemId::accessPointId.
    • K8s Binding: The driver creates a PV object with the generated VolumeId, which is then bound to the PVC.

    2. Node Publish Phase

    • Trigger: Kubelet schedules a Pod to a node.
    • Action: Kubelet calls the NodePublishVolume RPC on the Node Service.
    • Mounting: The Node Service extracts the fileSystemId and accessPointId from the VolumeId and uses efs-utils to mount the filesystem (using the access point and TLS options) at the target path.

    3. Deletion Phase

    • Trigger: User deletes the PVC.
    • Action: The csi-provisioner calls the DeleteVolume RPC on the Controller.
    • Cleanup: The Controller parses the accessPointId. If deleteAccessPointRootDir is enabled, it temporarily mounts the filesystem to delete the root directory and its contents before calling the EFS DeleteAccessPoint API and deleting the PV.