DOCSEARCHbyAlgolia
Submit Docs

K8ssandra Operator

repository·main·Indexed 42 days ago

https://github.com/k8ssandra/k8ssandra-operator

A Kubernetes operator designed to manage Apache Cassandra and DataStax Enterprise (DSE) across multiple clusters. It provides automated multi-region deployment, monitoring, repairs, and backups, and is distributed as a Helm chart.

Tokens
152.4K
Snippets
410
Records
595
Agent score
53%

What's inside K8ssandra Operator

  1. Overview of K8ssandra components

    main

    K8ssandra is a collection of open-source components designed to automate the management of Apache Cassandra® clusters on Kubernetes. It simplifies the provisioning and day-to-day operations of distributed Cassandra clusters by leveraging the Kubernetes control plane.

    Key capabilities include:

    • Automated Cassandra Management: Handles the deployment and upkeep of distributed NoSQL clusters.
    • Stargate Data API Gateway: Automates the deployment of Stargate, allowing developers to interact with Cassandra using REST, Document, and GraphQL APIs instead of the standard CQL protocol.
    • Multi-tenancy: Multiple K8ssandra installations can coexist within a single Kubernetes cluster by using Kubernetes namespaces as logical boundaries.
    • Simplified Lifecycle: Uses customizable Helm charts for easy installation and upgrades.

  2. Overview of K8ssandra Metrics Collection

    main

    K8ssandra provides observability for Apache Cassandra clusters by integrating with Prometheus (for metric storage) and Grafana (for visualization).

    Key components include:

    • Metrics Collector for Apache Cassandra (MCAC): A Java agent deployed within each Cassandra pod. It aggregates OS and Cassandra metrics along with diagnostic events. It is built on collectd and can track over 100k unique metric series per node.
    • Prometheus: A Prometheus instance (deployed via StatefulSet) that scrapes metrics from the MCAC agent.
    • Grafana: A visualization layer that uses preconfigured dashboards to display node-level, OS, and cluster-wide metrics.

    Supported Cassandra Versions: 2.2+ (including 2.2.X, 3.0.X, 3.11.X, and 4.0).

  3. Overview of K8ssandra Operator capabilities

    main

    k8ssandra-operator is a turnkey solution for managing Apache Cassandra and DataStax Enterprise (DSE) on Kubernetes. It enables the deployment of multiple Cassandra datacenters spanned across multiple Kubernetes clusters to provide geo-replication, low latency, and high availability.

    Key capabilities include:

    • Multi-Region Deployment: Automates multi-region deployments by enabling communication between multiple Kubernetes clusters. A single operator instance in a control plane cluster can manage many data plane DCs.
    • Monitoring: Provides Cassandra metrics via an agent added to the JVM. It integrates with Vector to stream metrics to destinations like Prometheus, Mimir, Elasticsearch, Kafka, or Redis. Metrics pipelines are configured via Kubernetes custom resources.
    • Repairs and Maintenance: Automates anti-entropy repairs using Reaper on a regular schedule, configurable via Kubernetes manifests.
    • Backup and Restore: Uses Medusa to back up Cassandra SSTables to cloud storage (S3, GCS, or Azure storage) using declarative Kubernetes manifests.
    • Advanced APIs: Supports deploying and configuring Stargate via Kubernetes manifests to provide GraphQL, REST, and Mongoose (Node.js) APIs.

  4. Overview of Medusa for Cassandra backup and restore

    main
    Medusa is a component deployed by the K8ssandra Operator to manage backup and restore operations for Apache Cassandra tables. It handles the complexities of distributed system backups, such as synchronizing snapshots across all nodes and correlating data files with remote storage. Medusa is managed via the Medusa Custom Resource Definition (CRD).

  5. Identify core operations managed by Cass Operator

    main

    The Cass Operator automates the lifecycle of Cassandra and DSE data platforms within Kubernetes. Key managed operations include:

    • Provisioning: Setting up datacenters and nodes.
    • Scaling: Increasing or decreasing the number of nodes.
    • Rolling Restarts: Performing restarts across the cluster without downtime.
    • Upgrades: Managing software version transitions.
    • Failure Remediation: Automatically handling container failures.

  6. What is Reaper and how does it work with K8ssandra?

    main

    Reaper is an open-source tool used to schedule and orchestrate anti-entropy repairs for Apache Cassandra clusters. It is deployed by the K8ssandra Operator via a Reaper Custom Resource Definition (CRD).

    Reaper maintains data consistency (anti-entropy) by splitting repair operations into evenly sized segments (subranges of tokens). This approach makes repairs safer and more reliable compared to traditional nodetool repair by:

    • Running only one segment per node at a time.
    • Terminating and rescheduling segments that take too long.
    • Replaying failed segments to handle transient failures.
    • Monitoring pending compactions to pause scheduling and prevent cluster overload.
    • Supporting incremental repairs (recommended for Cassandra 4.0+).
    • Consolidating token ranges with the same replicas into single segments to reduce vnode overhead (supported since Cassandra 3.0).

  7. Reference a remote data plane in a K8ssandraCluster

    main

    When defining a K8ssandraCluster resource that includes remote data planes, you can reference them within the DC list using the k8sContext field.

    • When using k8ssandra-client: You can simply refer to the data plane by the name of the ClientConfig resource.
    • When created manually: You must use the value found in spec.contextName of the ClientConfig resource, as it may differ from the metadata.name.

    Warning: Deleting a ClientConfig or its backing secrets will prevent the operator from managing any K8ssandraCluster resources that rely on those credentials.

  8. Compare dataSource and dataSourceRef for volume population

    main

    When configuring volume population in CassandraDatacenter, you can use either dataSource or dataSourceRef. dataSourceRef is the modern replacement for dataSource and offers more flexibility.

    Key Differences

    FeaturedataSourcedataSourceRef
    Supported ObjectsOnly VolumeSnapshot or PVCAny non-core API object or PVC
    Error HandlingIgnores disallowed values (drops them)Preserves all values; generates error on disallowed values
    Namespace ScopeOnly allows local objectsAllows objects in any namespaces (requires CrossNamespaceVolumeDataSource feature gate)

    Usage Rules

    • If both fields are non-empty, they must have the same value.
    • Backwards Compatibility: If namespace is not specified in dataSourceRef, and one field is empty while the other is non-empty, both will be set to the same value automatically.
    • Namespace Specificity: If namespace is specified in dataSourceRef, dataSource must be empty.
    • Feature Gates:
      • Using dataSourceRef requires the AnyVolumeDataSource feature gate (Beta).
      • Using the namespace field in dataSourceRef requires the CrossNamespaceVolumeDataSource feature gate (Alpha).

  9. Use shared MedusaConfiguration for multiple clusters

    main

    To avoid repeating storage settings across multiple K8ssandraCluster resources, you can create a MedusaConfiguration custom resource in the Control Plane cluster.

    Key behaviors:

    • The K8ssandraCluster definition can reference a configuration using medusaConfigurationRef.
    • Properties defined in the K8ssandraCluster storageProperties will override those in the MedusaConfiguration.
    • You must specify a prefix in the K8ssandraCluster definition to ensure multitenancy (preventing different clusters from overwriting each other's backups in the same bucket).
    # Shared Configuration
apiVersion: medusa.k8ssandra.io/v1alpha1
kind: MedusaConfiguration
metadata:
  name: medusaconfiguration-s3
spec:
  storageProperties:
    storageProvider: s3
    region: us-west-2
    bucketName: k8ssandra-medusa
    storageSecretRef:
      name: medusa-bucket-key

---

# Cluster Reference
apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
  name: demo
spec:
  medusa:
    medusaConfigurationRef:
      name: medusaconfiguration-s3
    storageProperties:
      prefix: demo

  10. Configure preStop lifecycle hooks for Cassandra initContainers

    main

    The preStop hook is called immediately before a container is terminated (e.g., due to an API request or liveness probe failure). The Pod's termination grace period begins before the hook executes. You can use one of the following handlers within CassandraDatacenter.spec.podTemplateSpec.spec.initContainers[index].lifecycle.preStop:

    • exec: Executes a specific command in the container. The command is not run inside a shell; to use shell features like pipes, you must explicitly call a shell (e.g., /bin/sh -c). An exit status of 0 is considered successful.
    • httpGet: Performs an HTTP GET request. You can specify the port, host, path, scheme, and custom httpHeaders.
    • sleep: Forces the container to sleep for a specified number of seconds.
    • tcpSocket: DEPRECATED. This is not supported and will cause lifecycle hooks to fail at runtime. Do not use this field.
    # Example of a preStop hook using exec
lifecycle:
  preStop:
    exec:
      command: ["/bin/sh", "-c", "echo cleaning up"]

  11. Connect to remote clusters using ClientConfig

    main

    To enable the K8ssandra Operator (control plane) to manage datacenters in remote Kubernetes clusters, you must provide access via a ClientConfig custom resource.

    A ClientConfig defines a remote cluster by providing a kubeconfig that the operator uses to remotely access that cluster.

    Workflow for Multi-Cluster Deployment:

    1. Install the K8ssandra Operator in your designated control plane cluster.
    2. Create and deploy ClientConfig resources for each remote data plane cluster to provide the operator with kubeconfig access.
    3. Deploy the K8ssandraCluster resource to the control plane cluster, referencing the remote clusters via the k8sContext field in the datacenter specifications.

  12. Use Ephemeral volumes in CassandraDatacenter

    main

    The ephemeral volume source represents a volume handled by a cluster storage driver where the lifecycle is tied to the pod. The volume is created before the pod starts and deleted when the pod is removed.

    When to use ephemeral volumes:

    • The volume is only needed while the pod is running.
    • You need features like restoring from snapshots or capacity tracking.
    • The storage driver supports dynamic volume provisioning via a PersistentVolumeClaim (PVC).

    Note: For volumes that must persist longer than an individual pod's lifecycle, use PersistentVolumeClaim or vendor-specific APIs instead.