DOCSEARCHbyAlgolia
Submit Docs

Capsule Documentation

repository·main·Indexed 24 days ago

https://github.com/projectcapsule/capsule

Capsule is a Kubernetes operator that simplifies multi-tenancy by introducing a 'Tenant' abstraction to group namespaces and enforce inherited policies, including RBAC, Quotas, and Network Policies. The documentation covers installation, Helm chart configuration for the operator and manager, webhook setup, OpenTelemetry tracing, and monitoring via Prometheus and Grafana.

Tokens
15K
Snippets
10
Records
53
Agent score
80%

What's inside Capsule

  1. Key features of Capsule

    main

    Capsule provides several capabilities for managing multi-tenant Kubernetes clusters:

    • Self-Service: Enables developers to provision cluster resources within assigned boundaries.
    • Preventing Clusters Sprawl: Reduces operational overhead by allowing multiple teams or departments to share a single cluster.
    • Governance: Uses Kubernetes Admission Controllers to enforce security best practices and policy requirements.
    • Resources Control: Prevents users from over-consuming resources through automated enforcement.
    • Native Experience: Operates using standard Kubernetes primitives without requiring custom binaries or additional management layers.
    • GitOps Ready: The entire system is declarative, making it compatible with GitOps workflows.
    • Bring Your Own Device (BYOD): Allows assigning dedicated compute, storage, and network resources to specific Tenants to prevent the 'noisy neighbor' effect.

  2. What is Capsule and how does it work?

    main

    Capsule is a multi-tenant and policy-based environment for Kubernetes. It solves the problem of 'cluster sprawl'—where organizations provision separate clusters for different teams to achieve isolation—by allowing multiple teams to share a single cluster securely.

    Core Abstraction: The Tenant

    Instead of managing individual Kubernetes Namespaces in isolation, Capsule introduces the Tenant abstraction. A Tenant is a lightweight grouping of Kubernetes Namespaces.

    How it provides isolation and governance

    • Aggregation: The Capsule Controller aggregates multiple namespaces into a single Tenant. Within a Tenant, users can share resources across their assigned namespaces.
    • Policy Inheritance: The Capsule Policy Engine ensures isolation between different Tenants. Policies defined at the Tenant level are automatically inherited by all namespaces within that Tenant. This includes:
      • Network and Security Policies
      • Resource Quotas
      • Limit Ranges
      • RBAC (Role-Based Access Control)
    • Self-Service: Once boundaries are set at the Tenant level, users can self-provision resources within their assigned limits without requiring cluster administrator intervention.

  3. Understand the role and responsibilities of a Capsule Maintainer

    main

    Maintainers are the governing body of the Capsule project, collectively known as the Maintainer Council.

    Key Responsibilities:

    • Repository Access: Maintainers have write access to the project GitHub repository and can merge their own patches or those from others.
    • Project Management: They collectively manage project resources and contributors.
    • Governance: They participate in public developer meetings and closed meetings (for security or Code of Conduct issues).
    • Decision Making: Most decisions follow "lazy consensus," but Maintainers can demand formal votes. A simple majority is typically required for decisions, while a 2/3 majority is required for specific actions like removing a Maintainer or modifying the Governance charter.

  4. How Capsule's multi-tenancy model works

    main

    Capsule implements multi-tenancy by aggregating multiple Kubernetes Namespaces into a lightweight abstraction called a Tenant.

    Core Components

    • Tenant: A grouping of Kubernetes Namespaces. Within a tenant, users can create namespaces and share assigned resources.
    • Capsule Operator: The central component providing multi-tenancy capabilities. It consists of:
      • Tenant Controller: Manages the lifecycle of tenants and reconciles Namespace-level objects like NetworkPolicy, LimitRange, ResourceQuota, and RoleBinding. It also handles object replication via the Tenant Resource API.
      • Policy Engine (Webhook Server): A part of the Tenant Controller that uses Kubernetes dynamic admission controllers (ValidatingWebhookConfiguration and MutatingWebhookConfiguration) to enforce tenant-level rules. It can prevent untrusted registries or non-allowed StorageClass usage and acts as a defaulter webhook for classes like IngressClass or RuntimeClass.
    • Capsule Proxy: An optional addon that acts as a Kubernetes API Server shim. It is a reverse proxy that decorates incoming requests with labelSelector query parameters to filter resources (Namespaces, IngressClass, StorageClass, etc.) based on Tenant ownership rather than standard RBAC.

    Policy Inheritance

    Policies defined at the Tenant level (Network/Security Policies, Resource Quotas, etc.) are automatically inherited by all Namespaces within that tenant, allowing users to operate autonomously without cluster administrator intervention.

  5. Use Capsule Proxy for filtered API access

    main

    The capsule-proxy acts as a middle proxy server that provides a Kubernetes API Server shim. It allows tools like kubectl or Kubernetes dashboards to interact with the cluster while only seeing resources relevant to the user's tenant.

    How it works

    The proxy decorates incoming requests with labelSelector query string parameters to filter the following resources:

    • Namespaces
    • IngressClass
    • StorageClass
    • PriorityClass
    • RuntimeClass
    • PersistentVolumes

    Benefits

    • No API Mangling: It does not change the upstream API responses, meaning it works with standard Kubernetes binaries without extra plugins.
    • Tenant-Level Permissions: Permissions are enforced at the Tenant Owner level rather than through traditional Kubernetes RBAC.

  6. Security Response Team responsibilities

    main
    The Maintainers appoint a Security Response Team to handle all reports of security holes and breaches. This team may consist of the Maintainer Council itself or a delegated team of at least two contributors. The Maintainers review the team's composition at least once a year. The team operates according to the project's security policy.

  8. How to become a Capsule Maintainer

    main

    To become a Maintainer of the Capsule project, you must demonstrate commitment through active participation in discussions, code/documentation reviews, and by contributing non-trivial pull requests that are successfully merged. You should also demonstrate the ability to write quality code/documentation, collaborate effectively with the team, and understand the project's purpose, codebase, and processes.

    The Process:

    1. An existing Maintainer must propose you by sending a message to all other existing Maintainers.
    2. A simple majority vote of the existing Maintainers is required for approval.
    3. Once approved, you will be granted the necessary GitHub rights to the project repository.

  9. Clean up the stress environment

    main

    Use the cleanup.sh script to remove resources from the stress environment. You can choose to delete only the generated workload or the entire cluster.

    • To delete only the generated workload (tenants, namespaces, etc.): use the --workload-only flag.
    • To delete the complete cluster: run the script without flags.

  10. Build and test Capsule

    main

    Use the provided Makefile to manage the build and testing lifecycle of the Capsule project.

    Building

    Build Docker images locally using ko. This will push the builds to your local Docker images.

    make ko-build-all

    Unit Testing

    Run the project's unit tests with:

    make test

    End-to-End (E2E) Testing

    E2E tests use the ginkgo framework. New changes should always include dedicated E2E tests.

    • Create a new KinD cluster and run tests: This creates a KinD cluster (defaulting to Kubernetes v1.20.7), builds/pushes the image, and executes tests.
      make e2e/v1.20.7
    • Run tests against an existing cluster: Use this if you do not want to create a new KinD cluster.
      make e2e-exec
    make ko-build-all
make test
make e2e/v1.20.7
make e2e-exec

  11. Apply tenant-level defaults and policies to workloads

    main

    Once a namespace belongs to a tenant, the Capsule controller automatically applies tenant-specific configurations to any new workloads or configurations applied to that namespace.

    Key Capabilities

    • Policy Enforcement: The controller inspects workloads and applies policies defined in the tenant configuration.
    • Tenant Defaults: Capsule allows for tenant-specific defaults (e.g., a specific StorageClass for a particular tenant) rather than relying solely on cluster-wide defaults. This provides a higher level of abstraction for tenant users.

  12. Integrate cert-manager for Certificate Generation

    main

    You can delegate certificate management to cert-manager by enabling certificate generation in the Helm values. When using this method, you must disable the built-in TLS creation and controller to allow cert-manager to handle the injection of the Validating and Mutating Webhooks' CA.

    Manual CRD Patching: Because Helm3 does not template CRDs, you must manually patch the tenants.capsule.clastix.io Custom Resource Definition to include the cert-manager.io/inject-ca-from annotation. This ensures cert-manager can inject the CA into the webhook configuration.

    helm upgrade --install capsule projectcapsule/capsule --namespace capsule-system --create-namespace \
  --set "certManager.generateCertificates=true" \
  --set "tls.create=false" \
  --set "tls.enableController=false"

    Required CRD Annotation:

    apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    cert-manager.io/inject-ca-from: capsule-system/capsule-webhook-cert
  name: tenants.capsule.clastix.io