DOCSEARCHbyAlgolia
Submit Docs

Solr Operator

repository·main·Indexed 42 days ago

https://github.com/apache/solr-operator

The official Kubernetes operator for managing Apache SolrCloud deployments. It provides automated lifecycle management, backups, and metrics, including support for Prometheus exporters and Zookeeper Operator integration. Includes Helm charts for deploying the operator and SolrCloud instances, with support for mTLS secure communication and custom resource definition (CRD) configuration.

Tokens
38.4K
Snippets
110
Records
160
Agent score
54%

What's inside Solr Operator

  1. Overview of Solr Operator

    main

    The Solr Operator is the official tool for managing Apache SolrCloud deployments within Kubernetes. It is built using the Kube Builder framework and is considered production-ready.

    Important Note on API Stability: The API version is currently in beta (v1beta1). While the operator provides backwards-compatible upgrade paths, minor versions may introduce backwards-incompatible API changes. Always verify compatibility before upgrading.

  2. Understand the Apache Solr Operator's role and scope

    main

    The Apache Solr Operator is a cluster-scoped Kubernetes controller designed to manage Apache Solr deployments. It is not the Solr search server itself; the search server's security and functionality are modeled separately in the apache/solr repository.

    Core Functionality

    The operator watches specific Custom Resources (CRs) and reconciles them by managing underlying Kubernetes objects. It acts as a privileged intermediary, converting low-privilege namespace actions (like creating a CR) into high-privilege cluster mutations (like managing Secrets or executing commands in pods).

    Managed Custom Resources (CRs)

    • SolrCloud
    • SolrBackup
    • SolrPrometheusExporter

  3. Identify the trust boundaries of the Solr Operator

    main

    When evaluating security or triaging findings, distinguish between these two primary trust boundaries:

    1. Tenant $\rightarrow$ Operator (The CR Boundary): A namespace tenant (user or CI) creates or patches a SolrCloud, SolrBackup, or SolrPrometheusExporter specification. The operator consumes this spec and acts upon it using its own high-privilege ServiceAccount. This is the primary area of concern: fields in the CR are considered attacker-controllable inputs to a privileged process.

    2. Operator $\rightarrow$ Solr (The HTTP Boundary): The operator acts as an authenticated client to the managed Solr pods via the HTTPS admin API (using basic-auth k8s-oper and optionally mTLS). In this boundary, the operator is the client trusting the Solr server.

    Note on Reachability: A finding is considered within the operator's model only if it is reachable by a CRD author who is not a cluster-admin, and the effect crosses from the tenant's authority into the operator's cluster authority.

    tenant ─(create/patch CR)─► [k8s API + RBAC] ─► operator reconcile loop
                                                    │ acts as operator SA (ClusterRole)
                                                    ├─► create/patch Secrets, ConfigMaps, Services
                                                    ├─► create/patch StatefulSets, Deployments, PDBs, Ingress, PVCs
                                                    ├─► pods/exec (create) into managed pods
                                                    └─► provision ZookeeperCluster (pravega CRD)
operator ─(HTTPS admin API, basic-auth k8s-oper, optional mTLS)─► managed Solr pods
   (TLS server-cert verification OFF by default: tls-skip-verify-server=true)

  4. Solr Image compatibility and custom images

    main

    The Solr Operator is guaranteed to work with official Solr images.

    Custom Images: You can use custom images if they are compatible with the official image structure. For Solr 9 and later, the Docker image creation is bundled within Solr. You can build an officially compliant image containing your custom additions by running ./gradlew docker within the Solr repository.

    Version Compatibility: Always check the Version Compatibility Matrix to ensure your chosen Solr version is supported by the Solr Operator and that the specific features you require are enabled for that version.

  5. Identify potential security risks in Custom Resource (CR) configuration

    main

    When designing or auditing SolrCloud Custom Resources, be aware of the 'confused-deputy' surface. The operator processes several tenant-controllable fields that name Kubernetes objects. If the operator does not strictly constrain these to the CR's own namespace, a CR author might attempt to access resources they shouldn't.

    Sensitive fields to audit include:

    • Secret references (e.g., basicAuthSecret, ZK ACL secrets, TLS keystore secrets, backup-repo credential secrets).
    • Volume sources.
    • Pod spec overrides (customSolrKubeOptions, volumes, security context, image).

    Security Note: The operator does not currently use an admission/validation webhook for CR validation; it relies on OpenAPI schema and cluster-level admission policies (like PSA, Gatekeeper, or Kyverno).

  6. How Solr backups work

    main

    Backups in the Solr Operator are managed through two distinct Custom Resource Definitions (CRDs) that work together:

    1. SolrCloud CRD: Used to define backup repositories. A repository is a metadata definition describing where and how backup data is stored (e.g., GCS, S3, or a Volume).
    2. SolrBackup CRD: Used to trigger the actual backup. It references a named repository defined in the SolrCloud spec, specifies which Solr collections to back up, and can optionally define a recurring schedule.

    To explore the full configuration schema, use: kubectl explain solrcloud.spec.backupRepositories and kubectl explain solrbackup.

  7. Handling failed or paused cluster operations

    main

    Cluster operations may pause and enter a retry queue if they exceed their timeout period or encounter errors (such as insufficient Kubernetes resources or replica placement constraints).

    Operation Timeouts

    • Scaling (Up or Down): 1 minute
    • Rolling restarts: 10 minutes

    Resolving Deadlocks and Failures

    When an operation is paused, the Solr Operator allows you to make changes to the cluster to resolve the underlying issue. Once your changes are applied, the operator will attempt to dequeue and retry the operation.

    Common Scenarios:

    • Insufficient Resources: If a Scale Up is paused due to lack of resources, you can decrease the resource requirements in the Pod Template. This triggers a Rolling Update which, once complete, allows the queued Scale Up to proceed.
    • Replica Placement Failures: If a Scale Down fails because replicas cannot be moved to other pods, you may need to scale the SolrCloud spec back to the current number of running pods. The operator will then trigger a "fake" Scale Up to balance replicas across the existing pods.

  8. Understand the security boundaries and scope of the Solr Operator

    main

    The Solr Operator's threat model is distinct from the Solr search server itself. When evaluating security or configuring the operator, keep the following scope boundaries in mind:

    • In-Scope: The Solr Operator logic, its Kubernetes controller behavior, and its management of SolrCloud resources.
    • Out-of-Scope: The Solr search server (managed APIs, SSRF, etc.), the Pravega ZK operator, and testing/example code. Security for the search server itself is covered by the Apache Solr threat model.
    • Actor Model: The primary adversary is typically a namespace tenant / CR author (a semi-trusted user who can create Custom Resources). A cluster-admin is considered fully trusted and is out of scope.
    • Dependency Trust: ZooKeeper (and the Pravega operator) are treated as trusted, out-of-model dependencies. The operator is responsible for managing ZK ACL security.

  9. Understand the Solr Operator security model and trust boundaries

    main

    The operator acts as a privileged intermediary between namespace tenants and the Kubernetes API.

    Trust Boundaries

    • CR Spec (Untrusted): The primary input is the Custom Resource (CR) spec. Because there are no admission/validation webhooks, the operator relies on OpenAPI schema constraints. Users can control pod/container overrides, volumes, env vars, and secret references.
    • Secrets (Namespace-Scoped): The operator reads secrets (like basicAuthSecret or ZK ACL secrets) from the same namespace as the resource being managed.
    • Operator Privileges: The operator uses a ClusterRole to manage resources, but it is designed to act only on the resource kinds and verbs granted to it by Kubernetes RBAC.

    Adversary Model

    • Namespace Tenant: A user who can create/patch CRs in a watched namespace but lacks the operator's cluster privileges. They may attempt to escalate privileges by referencing secrets or volumes outside their namespace.
    • Compromised Solr Pod: An attacker who has compromised a managed Solr pod might attempt to impersonate the Solr server to the operator (mitigated by enabling TLS verification).

  10. How cluster operation locks work

    main

    To prevent data corruption and ensure availability, the Solr Operator allows only one cluster operation to run at a time. This is enforced using annotations on the SolrCloud's StatefulSet.

    Lockable Operations

    • Managed Rolling Updates
    • Scaling Down with Replica Migrations
    • Scaling Up with Replica Migrations
    • Balancing Replicas Across Pods (triggered after Rolling Updates with Ephemeral Data or ScaleUp operations)

    Lock Implementation

    You can monitor the current state of locks and pending operations by inspecting the following annotations on the StatefulSet:

    • solr.apache.org/clusterOpsLock: Indicates the specific cluster operation currently holding the lock and executing.
    • solr.apache.org/clusterOpsRetryQueue: A queue of cluster operations that timed out or failed and are waiting to be retried in order.

  11. Upgrade Zookeeper Operator

    main

    The Solr Operator v0.3.0 requires a specific version of the Zookeeper Operator. If you are upgrading from v0.2.6 or earlier, you must upgrade the Zookeeper Operator to v0.2.9.

    If you use the Solr Operator Helm chart, it will attempt to deploy the correct version automatically. However, if you previously installed the Zookeeper Operator manually using the URL provided in the Solr Operator repository, you may need to delete the old resources first to allow Helm to manage them.

  12. Default Authorization rules in bootstrapped SolrCloud

    main

    When the operator bootstraps security, it creates three default users with specific roles and permissions:

    1. admin: Has admin and k8s roles. Can perform all actions including update, security-edit, and all operations.
    2. solr: Mapped to the users role. Has read access to collections but cannot add/update/delete documents.
    3. k8s-oper: Mapped to the k8s role. Used by the operator for cluster management (status, metrics, zookeeper status, ping, and replica balancing).

    The k8s role is also granted the collection-admin-edit permission to allow the operator to manage collections.