CSI Rclone Driver

The Rclone CSI Driver (CSI plugin name: rclone.csi.veloxpack.io) allows Kubernetes pods to mount cloud storage backends as persistent volumes using rclone. It supports over 50 storage providers, including Amazon S3, Google Cloud Storage, Azure Blob, Dropbox, and SFTP.

Key architectural advantages include:

• No External Dependencies: rclone is integrated directly as a Go library, so no rclone binary installation is required on the host.
• No Process Overhead: Direct library integration avoids subprocess spawning.
• Flexible Provisioning: Supports both dynamic volume provisioning via StorageClass and ephemeral/inline volumes defined directly in Pod specs.
• VFS Caching: High-performance caching with configurable options.

The driver is composed of three primary server components that implement the CSI specification:

1. Identity Server: Handles driver metadata and health checks via GetPluginInfo, GetPluginCapabilities, and Probe.
2. Controller Server: Manages the volume lifecycle (creation and deletion) and validates parameters via CreateVolume, DeleteVolume, and ValidateVolumeCapabilities.
3. Node Server: Manages the actual mounting and unmounting of rclone remotes on specific nodes via NodePublishVolume and NodeUnpublishVolume.

Key Design Note: This driver does not implement NodeStageVolume or NodeUnstageVolume because Rclone volumes do not require a staging step.

The driver supports template variables in the remotePath parameter to allow for dynamic directory structures based on Kubernetes metadata. This is useful for automatically organizing data by namespace or PVC name.

Supported variables:

• ${pvc.metadata.name}: The name of the PVC.
• ${pvc.metadata.namespace}: The namespace of the PVC.
• ${pv.metadata.name}: The name of the PV.

Example usage:

parameters:
  remote: "s3"
  remotePath: "my-bucket/${pvc.metadata.namespace}/${pvc.metadata.name}"

The driver supports three configuration methods, evaluated in the following priority order (highest to lowest):

1. PersistentVolume Configuration: Using volumeAttributes in the PV.
2. StorageClass parameters: Defined in the StorageClass spec.
3. Kubernetes Secrets: Referenced via csi.storage.k8s.io/node-publish-secret-name.

When multiple configuration sources are provided, the driver merges them using the following priority (later values override earlier ones):

1. Secrets (lowest priority)
2. StorageClass parameters (medium priority)
3. PersistentVolume volumeAttributes (highest priority)

The driver follows the Kubernetes CSI specification and is composed of three main server components:

1. Identity Server: Handles plugin metadata and health checks.
2. Controller Server: Manages the volume lifecycle (creation and deletion).
3. Node Server: Handles volume mounting and unmounting on individual Kubernetes nodes.

Key Design Characteristics

• No Staging: Rclone volumes do not require a staging step.
• Direct Rclone Integration: The driver uses the rclone Go library directly rather than calling the CLI.
• Remote Creation: It creates temporary remotes for each individual mount.
• VFS Caching: It leverages rclone's Virtual File System (VFS) to improve performance.
• Template Variable Support: Supports dynamic path substitution using PVC/PV metadata.

You can build custom deployment configurations by composing different components in a kustomization.yaml file. This allows you to mix and match features like metrics, services, and dashboards without using a pre-defined overlay.

Example: Metrics + Service + Dashboard (without Prometheus Operator):

namespace: veloxpack

resources:
  - ../../base

components:
  - ../../components/metrics-basic       # Enable metrics endpoint
  - ../../components/metrics-service     # Add Service
  - ../../components/metrics-dashboard   # Add Grafana dashboard

By default, all CSI resources are deployed to the veloxpack namespace.

Grafana Dashboards:

• The Grafana dashboard ConfigMap is created in the veloxpack namespace.
• If your Grafana sidecar is configured to watch all namespaces, it will automatically discover the dashboard.
• If your Grafana is restricted to a specific namespace (e.g., monitoring), you must manually copy the ConfigMap from veloxpack to that namespace.

To change the deployment namespace, edit the namespace: field in your kustomization.yaml file.

To run the Rclone CSI driver in a production environment, you must meet the following requirements:

• Kubernetes Version: 1.20 or later.
• CSI Node Driver Registrar: Must be present in the cluster.
• FUSE Support: Nodes must have FUSE support enabled for mounting volumes.
• Rclone Installation: Not required. The driver uses rclone as a Go library directly.

You can install the driver using Helm charts by adding the official repository.

1. Add and Update Repository

helm repo add csi-rclone https://veloxpack.github.io/csi-driver-rclone
helm repo update

2. Install the Driver

Install into a specific namespace (e.g., veloxpack):

helm install csi-rclone csi-rclone/csi-driver-rclone --namespace veloxpack

3. Verify Installation

helm list -n veloxpack
kubectl get pods -l app=csi-rclone-controller

To build the driver from source, ensure you have Go 1.21+, Docker, and rclone installed. Use the provided Makefile to compile the binary or build container images.

# Build the CSI Driver binary
make build

# Build the container image
make container

# Push the container image to a registry
make push