Cluster API Bootstrap Provider Talos

repository·main·Indexed 42 days ago

https://github.com/siderolabs/cluster-api-bootstrap-provider-talos

A Cluster API (CAPI) provider that automates the deployment of Talos-based Kubernetes nodes by generating Talos machine configurations and managing the necessary secrets for bootstrapping across various environments.

Tokens
3.2K
Snippets
10
Records
14
Agent score
53%

What's inside CABPT

  1. How to use CABPT via TalosConfigTemplate

    main

    CABPT is not used directly. Instead, it is consumed by TalosControlPlane (for control plane nodes) or MachineDeployment/MachinePool (for worker nodes) via the TalosConfigTemplate resource.

    Settings are passed through the spec.template.spec (for workers) or spec.controlPlaneConfig.controlplane (for control plane) fields.

    # Example for Control Plane
    apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
    kind: TalosControlPlane
    metadata:
      name: mycluster-cp
    spec:
      controlPlaneConfig:
        controlplane:
          generateType: controlplane
          talosVersion: v1.11
    
    # Example for Workers
    apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
    kind: TalosConfigTemplate
    metadata:
      name: mycluster-workers
    spec:
      template:
        spec:
          generateType: worker
          talosVersion: v1.9
  2. Understand CABPT reconciliation and status conditions

    main

    CABPT reconciles TalosConfig resources and manages several secrets:

    • <machine>-bootstrap-data: The generated machine configuration for a specific machine.
    • <cluster>-ca: The Kubernetes cluster CA.
    • <cluster>-talos: Cluster-wide shared secrets.
    • <cluster>-talosconfig: Client-side Talos API configuration.

    Status Conditions

    Monitor the TalosConfig/Status resource for these conditions:

    • DataSecretAvailable: Set to True when the machine configuration is stored in the <machine>-bootstrap-data Secret. This unblocks the infrastructure provider from booting the Machine.
    • ClientConfigAvailable: Set to True when the Talos client configuration is stored in the <cluster>-talosconfig Secret.

    If a condition fails, the Message and Reason fields will contain error details (e.g., DataSecretGenerationFailed).

  3. Deploy a new cluster on GCP

    main

    Use the sample manifests in config/samples/cluster-deployment/gcp as a template to deploy Talos clusters on GCP.

    1. Edit gcp-cluster.yaml, gcp-controlplane.yaml, and gcp-workers.yaml to include your specific GCP project ID and the GCP image you imported.
    2. Apply the manifests using kubectl.
    3. To interact with the nodes via osctl, you must retrieve the talosconfig and target the node's public IP.
    # Apply cluster manifests
    kubectl apply -f config/samples/cluster-deployment/gcp
    
    # Retrieve talosconfig for the first controlplane node
    kubectl get talosconfig -o yaml test1-controlplane-0 -o jsonpath='{.status.talosConfig}'
    
    # Target the node's public IP with osctl
    osctl config target $EXTERNAL_IP
  4. Install Cluster API Bootstrap Provider Talos (CABPT)

    main

    To install CABPT, use clusterctl init. It must be installed alongside the CACPPT (Control Plane Provider Talos) provider. Replace <infrastructure provider> with your specific infrastructure provider (e.g., aws, azure, vsphere).

    If you encounter a target namespace can't be defaulted error due to the GitHub organization rename from talos-systems to siderolabs, you must manually specify the provider URLs in your ~/.cluster-api/clusterctl.yaml file.

    clusterctl init --bootstrap talos --control-plane talos --infrastructure <infrastructure provider>
  5. Prepare the bootstrap cluster for GCP

    main

    Before deploying the Talos bootstrap provider, you must set up the GCP Infrastructure Provider in your management cluster.

    1. Install GCP Infrastructure Provider

    1. Clone the cluster-api-provider-gcp repository.
    2. Create a GCP Service Account and generate a JSON key file.
    3. Set the following environment variables and generate manifests:
      • GOOGLE_APPLICATION_CREDENTIALS: Path to your service account JSON file.
      • GCP_REGION: Your target GCP region (e.g., us-central1).
      • GCP_PROJECT: Your GCP project ID.
    4. Deploy the infrastructure components.
    5. Delete the kubeadm bootstrapper deployment as it is not required when using Talos.

    2. Install Talos Bootstrap Provider

    1. Clone the cluster-api-bootstrap-provider-talos repository.
    2. Apply the manifests using kustomize.
    # Generate GCP provider manifests
    export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
    export GCP_REGION=us-central1
    export GCP_PROJECT=my-gcp-project
    
    make generate-examples
    
    # Deploy infrastructure components
    kubectl create -f examples/_out/provider-components.yaml
    
    # Remove the unnecessary kubeadm controller
    kubectl delete deploy -n cabpk-system cabpk-controller-manager
    
    # Install Talos Bootstrap Provider
    kustomize build config/default/ | kubectl apply -f -
  6. Retrieve the `talosconfig` for a cluster

    main

    CABPT stores the generated client-side talosconfig as a secret named <cluster-name>-talosconfig in the cluster's namespace. Use the following commands to retrieve it and merge it into your local talosctl configuration:

    1. Extract and decode the secret.
    2. Merge it into your local config.
    3. Verify access.
    kubectl get secret --namespace <cluster-namespace> <cluster-name>-talosconfig -o jsonpath='{.data.talosconfig}' | base64 -d > cluster-talosconfig
    talosctl config merge cluster-talosconfig
    talosctl -n <IP> version
  7. Run integration tests for CABPT

    main

    To run the integration tests, you must have an existing Talos cluster (or any other Kubernetes cluster) available. The tests require a kubeconfig file located at the root of the repository and a release manifest generated with a specific tag. The tests are designed to clean up after themselves, allowing for repeated execution against the same cluster.

    Prerequisites

    1. Cluster: A running Talos or Kubernetes cluster.
    2. Kubeconfig: A kubeconfig file placed at the root of the repository. For Talos clusters, you can generate this using:
      talosctl -n <IP> kubeconfig -f kubeconfig
    3. Release Manifests: A release must be created with a fixed tag to ensure the CRDs match the test expectations:
      make release-manifests TAG=v0.6.0

    Execution

    Run the following command to execute the test suite (replace v0.6.0 with your target tag):

    make test TAG=v0.6.0
    talosctl -n <IP> kubeconfig -f kubeconfig
    make release-manifests TAG=v0.6.0
    make test TAG=v0.6.0
  8. Configure provider URLs in clusterctl.yaml

    main

    If clusterctl init fails to fetch provider components, add the following configuration to ~/.cluster-api/clusterctl.yaml to explicitly define the component URLs for the talos bootstrap provider, talos control plane provider, and sidero infrastructure provider.

    providers:
      - name: "talos"
        url: "https://github.com/siderolabs/cluster-api-bootstrap-provider-talos/releases/latest/bootstrap-components.yaml"
        type: "BootstrapProvider"
      - name: "talos"
        url: "https://github.com/siderolabs/cluster-api-control-plane-provider-talos/releases/latest/control-plane-components.yaml"
        type: "ControlPlaneProvider"
      - name: "sidero"
        url: "https://github.com/siderolabs/sidero/releases/latest/infrastructure-components.yaml"
        type: "InfrastructureProvider"
  9. Apply configuration patches to Talos nodes

    main

    You can customize generated machine configurations using strategicPatches. This allows you to override any field in the Talos machine configuration.

    spec:
      generateType: controlplane
      talosVersion: v1.14
      strategicPatches:
        - |
          apiVersion: v1alpha1
          kind: UnattendedInstallConfig
          provisioning:
            diskSelector:
                match: disk.dev_path == "/dev/sda"
        - |
          # disable Flannel CNI
          apiVersion: v1alpha1
          kind: KubeFlannelCNIConfig
          $patch: delete
  10. Configure TalosConfigTemplate fields

    main

    The following fields are available in TalosConfigTemplate and TalosConfig resources:

    • generateType: Determines the Talos machine configuration type to generate. Options: controlplane, worker, init (deprecated), or none (for user-supplied configuration).
    • talosVersion: The version of Talos to generate configuration for (e.g., v1.11). It is recommended to set this explicitly. CABPT defaults to the latest supported version.
    • configPatches (optional): A set of machine configuration patches to apply to the generated configuration.
    • data (only for generateType: none): The raw user-supplied machine configuration (can be generated via talosctl gen config).
    • hostname (optional): Configures the hostname. If source is set to MachineName, the hostname is set to the Machine name (not supported with MachinePool).