Provider Kubernetes

repository·main·Indexed 41 days ago

https://github.com/crossplane-contrib/provider-kubernetes

A Crossplane provider that enables the deployment and management of arbitrary Kubernetes resources by wrapping Kubernetes manifests in Crossplane 'Object' resources. It supports Server-side apply (SSA) for drift detection and field management, and offers enhanced features such as fine-grained resource management policies and generic references for resolving runtime values and managing installation/uninstallation order.

Tokens
5.6K
Snippets
11
Records
22
Agent score
51%

What's inside Provider Kubernetes

  1. Use Enhanced Provider Kubernetes features

    main

    The enhanced version of Provider Kubernetes (currently available in the morningspace/provider-kubernetes fork) introduces two key capabilities for managing arbitrary Kubernetes resources:

    1. Fine-grained Resource Management Policy: Allows you to instruct the provider on how to handle the lifecycle (Observe, Create, Update, Delete) of a resource.
    2. Generic References: Allows you to define dependencies between resources to retrieve values from dependent resources at runtime and guarantee specific installation and uninstallation orders.
  2. How the Object resource and external resources relate

    main

    The provider-kubernetes provider acts as a bridge between Crossplane and target Kubernetes clusters.

    1. Definition: You create an Object resource in Crossplane.
    2. Transformation: The provider takes the spec.forProvider.manifest and transforms it into the actual Kubernetes resource.
    3. Reconciliation: The provider checks if the external resource exists on the target cluster (defined by spec.providerConfigRef). If it does not exist, the provider creates it.
    4. Observation: The provider watches the external resource and updates the Object resource's status.atProvider.manifest field with the current runtime state of the target resource.
  3. Use Provider Kubernetes to implement advanced resource features

    main
    The enhanced Provider Kubernetes serves as a platform for implementing features that are not yet available in Crossplane core, such as advanced resource references. This allows developers to use cutting-edge functionality without forking the Crossplane core, making it easier to stay aligned with official Crossplane releases and upgrades.
  4. Configure Resource Management Policy

    main

    The managementPolicy field in an Object resource allows you to control the provider's interaction with the Kubernetes resource. This is useful for managing 'unmanaged external resources'—resources that are created/updated by third-party controllers (like OLM) but should be deleted by the provider when the managed Object is removed.

    Management PolicyObserveCreate & UpdateDelete
    DefaultYYY
    ObserveCreateUpdateYYN
    ObserveDeleteYNY
    ObserveYNN

    Note on DeletionPolicy vs managementPolicy: Crossplane's runtime DeletionPolicy (Orphan or Delete) and the provider's managementPolicy are separate. If both are defined, the Crossplane runtime policy is evaluated first:

    • If DeletionOrphan is set, the resource will not be deleted by the provider regardless of the managementPolicy.
    • If DeletionDelete is set, the provider's managementPolicy determines if the resource is actually deleted.
    apiVersion: kubernetes.crossplane.io/v1alpha1
    kind: Object
    metadata:
      name: csv-kong
    spec:
      managementPolicy: ObserveDelete
      forProvider:
        manifest:
          apiVersion: operators.coreos.com/v1alpha1
          kind: ClusterServiceVersion
          metadata:
            namespace: operators
  5. How Server-side apply (SSA) works in provider-kubernetes

    main

    By default, provider-kubernetes uses Kubernetes Server-side apply (SSA) to manage and sync resources.

    Core Mechanics

    • Intent Definition: The desired state of a Kubernetes object is defined in the .spec.forProvider.manifest of an Object Managed Resource (MR) (either kubernetes.crossplane.io/v1alpha1 for cluster-scoped or kubernetes.m.crossplane.io/v1alpha1 for namespace-scoped resources).
    • Field Ownership: The provider uses a specific field manager named provider-kubernetes/<object-mr-name> to apply changes.
    • Drift Detection: The provider only syncs and detects drift for the specific fields it manages (those explicitly defined in the .spec.forProvider.manifest) according to Kubernetes SSA mechanics.
    • Conflict Resolution: Following Kubernetes recommendations, provider-kubernetes forces conflicts if they occur during field management.
    WARN

    Avoid having other external controllers managing the same fields as the Object MR, as this can cause race conditions over field ownership.

  6. Managing the same k8s resource with multiple Object MRs

    main

    While discouraged, you can use multiple Object MRs to target the same Kubernetes resource, provided they manage disjoint field sets. Each MR must manage different fields to avoid racing on field ownership.

    Using Partial Manifests

    You can use an Object MR to partially manage an existing resource (created by another actor) by providing a partial manifest. These manifests are not valid standalone resources and cannot be used for creation.

    To use partial manifests correctly, follow these requirements:

    1. Lead Resource: Designate one Object MR (or an existing external resource) as the "lead" to handle the full lifecycle (creation/deletion). This lead must contain a valid, complete manifest.
    2. Management Policies: For all MRs providing partial manifests, set spec.managementPolicies to ["Observe", "Update"] only.

    Limitations of Partial Manifests

    • Creation: Partial manifests will remain in a Synced: False state until the target Kubernetes resource already exists.
    • Deletion: Deleting an Object MR with a partial manifest will not delete the target Kubernetes resource.
    • Cleanup: Currently, partial MRs will not remove the specific fields they manage when the MR is deleted.
  7. Manage arbitrary Kubernetes resources with Enhanced Provider Kubernetes

    main

    The enhanced Provider Kubernetes allows you to manage arbitrary Kubernetes resources by representing them as Crossplane Object resources. This is particularly useful for integrating Kubernetes-native backends like OLM (Operator Lifecycle Manager) or Helm into your Crossplane workflows.

    By using Object resources to represent the Custom Resources (CRs) watched by these backends, you can:

    • Drive backend installations/updates via Crossplane.
    • Apply fine-grained management policies.
    • Use resource references to resolve metadata at runtime.
    • Centrally manage the installation and uninstallation order of various components.
  8. Manage Install and Uninstall Order with References

    main

    The enhanced provider uses spec.references to manage the lifecycle order of resources:

    Install Order

    The provider blocks operations on a resource until all its referenced resources are resolved. By creating a dependency chain (e.g., Resource C references Resource B, which references Resource A), you guarantee that resources are created in the correct order.

    Uninstall Order

    To prevent 'garbage' resources from remaining in the cluster (e.g., a parent resource being deleted before a child that depends on it), the provider automatically adds finalizers to dependent resources when a reference is resolved.

    When a managed resource is deleted, the provider removes the finalizers from its dependents in the correct reverse order, ensuring a clean uninstallation.

  9. Use Generic References to resolve runtime values

    main

    Generic References allow an Object to resolve values from other managed resources at runtime. This is essential when a resource's name or attribute (like a generated ID) is not known until the dependency is created.

    You define these in the spec.references field using a syntax similar to Crossplane patches. The provider resolves these references in order; if any reference fails to resolve, reconciliation fails.

    apiVersion: kubernetes.crossplane.io/v1alpha1
    kind: Object
    metadata:
      name: csv-kong
    spec:
      references:
      - fromObject:
          apiVersion: kubernetes.crossplane.io/v1alpha1
          kind: Object
          name: sub-kong
          fieldPath: status.atProvider.manifest.status.currentCSV
        toFieldPath: spec.forProvider.manifest.metadata.name
      managementPolicy: ObserveDelete
      forProvider:
        manifest:
          apiVersion: operators.coreos.com/v1alpha1
          kind: ClusterServiceVersion
          metadata:
            namespace: operators
  10. Switching from patch-based syncer to SSA

    main

    If you have existing Object MRs that were created using the legacy patch-based syncer, provider-kubernetes automatically handles the migration to the SSA-based syncer.

    During this transition, fields previously managed by the patch-based syncer are transferred to the SSA-based syncer's field manager. Note that .metadata.managedFields entries belonging to other external managers are not modified during this process.

  11. Manage arbitrary Kubernetes resources with the Object resource

    main

    The provider-kubernetes provider allows you to manage any Kubernetes resource by wrapping its manifest inside a Crossplane Object resource.

    Key Concepts

    • Managed Resource: The Crossplane Object resource itself, which lives in the cluster where the provider is running.
    • External Resource: The actual target Kubernetes resource (e.g., a Namespace, Deployment, etc.) that the provider manages on a target cluster.
    • Manifest Wrapping: You define the target resource's apiVersion, kind, and metadata within spec.forProvider.manifest.
    • Status Monitoring: Once reconciled, the provider watches the external resource and copies its runtime manifest and status back to the Object resource at status.atProvider.manifest.

    Naming Convention

    In the spec.forProvider.manifest.metadata.name field, the name is optional. If omitted, the provider defaults to using the metadata.name of the Crossplane Object resource.

    apiVersion: kubernetes.crossplane.io/v1alpha1
    kind: Object
    metadata:
      name: sample-namespace
    spec:
      forProvider:
        manifest:
          apiVersion: v1
          kind: Namespace
          metadata:
            # name in manifest is optional and defaults to Object name
            # name: some-other-name
            labels:
              example: "true"
      providerConfigRef:
        name: kubernetes-provider