SeaweedFS CSI Driver

To ensure the Kubernetes scheduler places pods with SeaweedFS PVCs on nodes where the driver is actually running, you must configure topologyKeys. When configured, the driver reports the labels of the nodes it runs on as accessible topology. This populates CSINode.spec.drivers[].topologyKeys, which in turn adds nodeAffinity to provisioned Persistent Volumes (PVs).

Configuration Options

You can configure topology keys in two ways:

1. Helm Chart: Set the topologyKeys field in values.yaml.
2. CLI Arguments: Use the --topologyKeys= flag on the following containers:
   • csi-seaweedfs-plugin (running in a DaemonSet)
   • seaweedfs-csi-plugin (running in a Deployment)

Best Practice: The CSI spec prefers a single key prefix across all topology keys. Use keys like topology.kubernetes.io/zone and topology.kubernetes.io/region together.

Restricting Volume Provisioning via StorageClass

Once topology keys are configured, you can use a StorageClass to restrict where volumes are provisioned using allowedTopologies.

# Helm values.yaml example
topologyKeys:
  - topology.kubernetes.io/zone

# StorageClass example to restrict to a specific zone
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: seaweedfs-zone-a
provisioner: seaweedfs.csi.seaweedfs.io
allowedTopologies:
  - matchLabelExpressions:
      - key: topology.kubernetes.io/zone
        values:
          - zone-a

To use SeaweedFS as a CSI provider in HashiCorp Nomad, you must deploy two distinct components that share the same csi_plugin ID (seaweedfs):

1. Controller (seaweedfs-csi-controller.hcl): A single service job that manages the volume lifecycle (RPCs). It communicates with the SeaweedFS filer. You must run this to enable nomad volume create commands.
2. Node (seaweedfs-csi.hcl): A system job that runs on every worker node. It handles staging and publishing volumes to allocations using a seaweedfs-mount sidecar via a shared unix socket.

Note: If you do not run the controller, you cannot use nomad volume create. Instead, you must manually pre-create the bucket or directory in SeaweedFS and use nomad volume register to register the existing volume.

export NOMAD_ADDR=http://nomad.service.consul:4646

# Start CSI controller (one instance)
nomad run seaweedfs-csi-controller.hcl

# Start CSI node (one per worker)
nomad run seaweedfs-csi.hcl

# Verify plugin status
nomad plugin status seaweedfs

# Create a volume
nomad volume create example-seaweedfs-volume.hcl

# Run an application using the volume
nomad run example-seaweed-app.hcl

The driver supports both dynamic and static provisioning.

Dynamic Provisioning

By default, the driver creates a separate folder (/buckets/<volume-id>) and a separate collection (volume-id) for every request. You can customize this by creating a StorageClass with specific parameters:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: seaweedfs-special
provisioner: seaweedfs-csi-driver
parameters:
  collection: mycollection
  replication: "011"
  diskType: "ssd"

Static Provisioning

To access a specific folder from different pods with ReadWriteMany access, create a PersistentVolume (PV) manually and then bind a PersistentVolumeClaim (PVC) to it.

Example PV:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: seaweedfs-static
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Gi
  csi:
    driver: seaweedfs-csi-driver
    volumeHandle: dfs-test
    volumeAttributes:
      collection: default
      replication: "011"
      path: /path/to/files
      diskType: "ssd"
    readOnly: true
  persistentVolumeReclaimPolicy: Retain
  volumeMode: Filesystem

Example PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: seaweedfs-static
spec:
  storageClassName: ""
  volumeName: seaweedfs-static
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

To build the plugin manually, you can use the following sequence of commands. This process builds a Docker image, exports its filesystem to a rootfs directory, and creates a Docker plugin from that directory.

Note: The build script expects arguments for [PREFIX] [VERSION] [TAG] [PLUGIN_NAME] [ARCH].

#!/bin/bash
USAGE="Usage: ./build.sh [PREFIX] [VERSION] [TAG] [PLUGIN_NAME] [ARCH]"

if [[ "$1" == "-h" ]]; then
  echo "$USAGE"
  exit 0
fi

VERSION=${2:-latest}
ARCH=${5:-linux/amd64}
PLUGIN_NAME=${4:-swarm-csi-swaweedfs}
PLUGIN_TAG=${3:-v1.2.0}
PREFIX=${1:-gradlon}

docker build --platform ${ARCH} --build-arg BASE_IMAGE=chrislusf/seaweedfs-csi-driver:${VERSION} --build-arg ARCH=$ARCH -t seawadd-csi_tmp_img .
mkdir -p ./plugin/rootfs
cp config.json ./plugin/
docker container create --name seawadd-csi_tmp seawadd-csi_tmp_img 
docker container export seawadd-csi_tmp | tar -x -C ./plugin/rootfs
docker container rm -vf seawadd-csi_tmp 
docker image rm seawadd-csi_tmp_img 

docker plugin disable ${PREFIX}/swarm-csi-swaweedfs:v1.2.0
docker plugin rm ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} 2> /dev/null || true
docker plugin create ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG} ./plugin
docker plugin push ${PREFIX}/${PLUGIN_NAME}:${PLUGIN_TAG}
rm -rf ./plugin/

For correct operation, both Volume-Servers and the CSI-Driver-Node must have the DataCenter locality option correctly set. This ensures the driver can correctly identify data locality.

Automatic injection in Kubernetes

Because Kubernetes' DownwardAPI does not support mapping node labels (containing region/zone information) directly to environment variables, you must use a workaround if you want automatic injection.

Using KubeMod: If KubeMod is installed in your cluster, you can automate this via the Helm chart configuration:

1. Enable injection in values.yaml: node.injectTopologyInfoFromNodeLabel.enabled: true.
2. Specify which labels to grab using node.injectTopologyInfoFromNodeLabel.labels.

It is recommended to use well-known Kubernetes topology labels (e.g., topology.kubernetes.io/zone or topology.kubernetes.io/region) to avoid configuration errors.

The recommended way to install the driver is using Helm.

1. Add the repository:

   helm repo add seaweedfs-csi-driver https://seaweedfs.github.io/seaweedfs-csi-driver/helm
   helm repo update seaweedfs-csi-driver

2. Install with a single Filer: Clone the project and install by setting the seaweedfsFiler value:

   git clone https://github.com/seaweedfs/seaweedfs-csi-driver.git
   helm install --set seaweedfsFiler=<filerHost:port> seaweedfs-csi-driver ./deploy/helm/seaweedfs-csi-driver

3. Install with multiple Filers: To provide multiple filer addresses, use a comma-separated list (ensure you escape commas if using --set in some shells):

   helm install seaweedfs-csi-driver ./deploy/helm/seaweedfs-csi-driver/ \
     --namespace seaweedfs-csi-driver \
     --set seaweedfsFiler="<filerHost:port>\,<filerHost:port>\,<filerHost:port>"

4. Uninstall:

   helm uninstall seaweedfs-csi-driver