Kubernetes Secrets Store CSI Driver

The Kubernetes Secrets Store CSI Driver (secrets-store.csi.k8s.io) integrates external enterprise-grade secrets stores with Kubernetes using the Container Storage Interface (CSI). It allows you to mount multiple secrets, keys, and certificates from external providers directly into your pods as a volume. Once the volume is attached, the data is mounted into the container's file system.

Key features include:

• Mounting secrets/keys/certs to pods using a CSI Inline volume.
• Supporting multiple secrets store objects as a single volume.
• Supporting multiple secrets store providers simultaneously in the same cluster.
• Enabling pod portability via the SecretProviderClass CRD.
• Support for both Linux and Windows containers.
• Support for synchronization with native Kubernetes Secrets.

The driver provides the following stable core functionalities:

• Multiple Providers: Support for various external secrets store providers.
• Pod Portability: Uses the SecretProviderClass Custom Resource Definition (CRD) to ensure pods can be moved across clusters easily.
• CSI Inline Volumes: Mounts secrets, keys, and certs to pods using CSI Inline volumes.
• Volume Aggregation: Ability to mount multiple secrets store objects as a single volume.
• Cross-Platform Support: Works with both Linux and Windows containers.

The Kubernetes Secrets Store CSI Driver (secrets-store.csi.k8s.io) integrates external enterprise-grade secrets stores with Kubernetes using the Container Storage Interface (CSI) standard.

It allows Kubernetes pods to mount multiple secrets, keys, and certificates from external providers as a volume. Once the volume is attached, the data is mounted directly into the container's file system.

You can configure the Secrets Store CSI Driver to periodically update the pod mount and any synced Kubernetes Secrets with the latest content from the external secrets store.

Important Note: The CSI driver does not restart the application pods when secrets are rotated. It only updates the pod mount and the Kubernetes Secret, behaving similarly to how Kubernetes handles updates to secrets mounted as volumes.

Depending on how your application accesses secret data, you must choose the correct combination of features to ensure it receives updated values:

1. Mounted Kubernetes Secret Volume: Use auto rotation + Sync K8s secrets feature. The application must watch for changes in the mounted Kubernetes Secret volume. When the CSI Driver updates the Kubernetes Secret, the volume contents are updated automatically.
2. Container Filesystem: Use the rotation feature. The application must watch for file changes in the volume mounted directly by the CSI driver.
3. Kubernetes Secret as Environment Variables: The pod must be restarted to pick up new values. To automate this, use a tool like Reloader to watch the synced Kubernetes Secret and trigger rolling upgrades on your pods.

Containers using a subPath volume mount will not receive secret updates even if the Enable Secret autorotation feature is enabled.

This occurs because the CSI driver uses an atomic writer that relies on symlinks to update file content. When a secret is rotated, the symlink is updated, but the specific file bind-mounted via subPath remains pointed at the old version.

To ensure secrets are updated, you must either:

1. Restart the pod.
2. Avoid using subPath for volume mounts and instead mount the entire volume directory.

    volumeMounts:
    - mountPath: /app/spapi/settings.ini
      name: app-config
      subPath: settings.ini
...
  volumes:
  - csi:
      driver: secrets-store.csi.k8s.io
      readOnly: true
      volumeAttributes:
        secretProviderClass: app-config
    name: app-config

The Secrets Store CSI Driver facilitates the mounting of external secrets into Kubernetes pods.

Workflow:

1. Pod Start/Restart: The driver communicates with a provider via gRPC to retrieve secret content from an external Secrets Store, as specified in a SecretProviderClass resource.
2. Mounting: The secret contents are written to a volume mounted in the pod as a tmpfs (on Linux), ensuring secrets are kept in memory and not persisted to disk.
3. Pod Deletion: When the pod is deleted, the corresponding volume is automatically cleaned up.

Note on Persistence:

• Linux: Uses tmpfs to prevent secret material from being persisted to disk. To ensure this, ensure failSwapOn is set to true (the default) to prevent secrets from being written to disk via swap memory.
• Windows: Secrets are written to the node's filesystem, which may be persistent storage.

When evaluating the driver, consider the following security risks associated with how secrets are consumed:

• Filesystem access: If secrets are mounted on the filesystem, application vulnerabilities like directory traversal attacks increase the risk of attackers reading secret material.
• Environment variables: If secrets are consumed via environment variables, misconfigurations (e.g., enabling debug endpoints or using dependencies that log process environment details) may leak secrets.
• Kubernetes Secret sync: If syncing material to Kubernetes Secret objects, ensure that the access controls on the Kubernetes Secret data store are sufficiently narrow in scope.

For the highest security, consider directly integrating with a purpose-built secrets API.

The Secrets Store CSI Driver is deployed as a daemonset. Each driver pod contains the following containers:

• node-driver-registrar: A sidecar (provided by the Kubernetes CSI team) that registers the driver with Kubelet via a unix domain socket.
• secrets-store: The core component that implements the CSI Node service gRPC services. It handles mounting and unmounting volumes during pod lifecycle events.
• liveness-probe: A sidecar (provided by the Kubernetes CSI team) that monitors the health of the driver and reports to Kubernetes for automatic restarts if issues are detected.

The following features are currently in Alpha and are considered unstable. Users of these features should carefully consult the upgrade instructions during every upgrade:

• Auto rotation: Automatically rotates mounted contents and synced Kubernetes secrets.
• Sync with Kubernetes Secrets: Synchronizes external secrets into native Kubernetes Secret objects.