DOCSEARCHbyAlgolia
Submit Docs

Talos Terraform Provider

repository·main·Indexed 42 days ago

https://github.com/siderolabs/terraform-provider-talos

A Terraform provider for managing Talos Linux resources, automating the configuration and lifecycle of Talos-managed infrastructure. It supports ephemeral resources for managing machine secrets and kubeconfigs without persisting sensitive data to the Terraform state, integrating with secret managers like HashiCorp Vault. Includes data sources such as talos_client_configuration and talos_cluster_health for cluster readiness checks.

Tokens
29.9K
Snippets
61
Records
113
Agent score
55%

What's inside Talos Terraform Provider

  1. Use the Talos provider to manage Talos clusters

    main

    The Talos provider is designed to automate the lifecycle of a Talos cluster. It enables you to:

    • Generate configurations: Create the necessary configuration files for a Talos cluster.
    • Apply configurations: Deploy these configs to Talos nodes and bootstrap nodes.
    • Monitor health: Check the health status of the cluster.
    • Retrieve credentials: Automatically fetch kubeconfig and talosconfig files for cluster access.

    For complete usage examples across different environments, refer to the official Sidero Labs examples repository.

  2. Use the talos_machine_secrets ephemeral resource

    main

    The talos_machine_secrets resource is an ephemeral resource used to generate machine secrets for a Talos cluster.

    Important: Because it is ephemeral, it does not persist these secrets in the Terraform state. This is designed for generating secrets that are then passed to other resources (like Talos machine configurations) without leaving sensitive material in your state files.

  3. Why a Secret Manager is required for ephemeral resources

    main

    When using ephemeral "talos_machine_secrets", you must integrate with a secret manager (like HashiCorp Vault).

    Warning: Do not generate ephemeral "talos_machine_secrets" directly without persisting them to a secret manager. If you do not persist them, a new set of secrets will be generated on every terraform plan or apply. This leads to:

    • Dependent resources constantly showing changes.
    • Loss of cluster access when secrets regenerate.
    • Non-deterministic infrastructure state.

  4. How Kubernetes component image management works with talos_machine

    main

    When using talos_machine alongside talos_cluster, set ignore_kubernetes_upgrade_drift = true to prevent talos_machine from interfering with the graceful Kubernetes upgrade process managed by upgrade-k8s.

    Why use this?

    upgrade-k8s performs component-by-component, node-by-node upgrades with health gating. If talos_machine attempts to reconcile the Kubernetes component images (like kubelet or apiServer) in parallel, it bypasses these safety procedures.

    Behavior with ignore_kubernetes_upgrade_drift = true:

    • Drift in the five specific Kubernetes component image fields is not detected by talos_machine.
    • kubernetes_version in talos_machine_configuration and talos_cluster.kubernetes_version can be bumped in the same terraform apply without conflict.
    • A registry change (e.g., switching to a private mirror) is still detected as drift because the hash changes.
    • kubernetes_version in talos_machine_configuration still controls the version used during bootstrap for new nodes.

    Note: This is an experimental feature. If upgrade-k8s adds new managed image fields in future Talos versions, this attribute may need to be updated.

  5. How to use Ephemeral Resources with Secret Managers

    main

    Using ephemeral "talos_machine_secrets" alone is dangerous because it generates new secrets on every Terraform run, which will break your cluster access and cause non-deterministic infrastructure.

    The Correct Pattern:

    1. Generate secrets once using an ephemeral resource.
    2. Store them immediately in a secret manager (e.g., Vault, AWS Secrets Manager).
    3. Retrieve them ephemerally from the secret manager for subsequent runs.

    This ensures secrets remain stable across runs while never being written to the Terraform state file.

  6. Use Write-Only Attributes for Secret-Free Operations

    main

    To prevent secrets from leaking into Terraform state when using ephemeral data, the Talos provider provides "write-only" (_wo) attributes. When you use these attributes, the provider ensures that the sensitive values are never persisted in the state file.

    Key write-only attributes:

    • machine_configuration_input_wo: Use this on the talos_machine_configuration_apply resource instead of machine_configuration_input (requires Terraform 1.11+).
    • client_configuration_wo: Used on talos_machine_configuration_apply to pass client credentials without storing them.

    When using these _wo variants, the corresponding computed output (e.g., machine_configuration) will automatically be set to null in the Terraform state.

  7. How to use Ephemeral Resources safely with Secret Managers

    main

    Using ephemeral "talos_machine_secrets" alone is dangerous because it generates new secrets on every Terraform run, which will cause node reconfiguration needs and loss of cluster access.

    The Correct Pattern:

    1. Generate secrets once using an ephemeral resource.
    2. Store them immediately in a secret manager (e.g., Vault, AWS Secrets Manager).
    3. Retrieve them ephemerally from the secret manager for subsequent runs to ensure stability and deterministic infrastructure.

  8. Ensure deterministic secret behavior using `data_json_wo_version`

    main

    To prevent Talos from regenerating new machine secrets on every terraform apply, you must use a hardcoded version key in your Vault resource.

    When using vault_kv_secret_v2, set data_json_wo_version = 1 (or any fixed integer).

    How it works:

    • First Run: The secret does not exist in Vault. Terraform generates the secrets and stores them as version 1.
    • Subsequent Runs: Terraform checks the existing secret. Since the version matches the hardcoded value, no update is triggered. Because no update is needed, the ephemeral "talos_machine_secrets" block is not re-evaluated, preventing the generation of new, different secrets.

    If you see resources changing on every plan, ensure this version key is properly hardcoded.

  9. How to upgrade Kubernetes in Talos

    main

    Upgrading Kubernetes is driven by changing the kubernetes_version argument in the talos_cluster resource. This triggers Talos's upgrade-k8s procedure, which performs a sequenced upgrade:

    1. Pre-pulls images.
    2. Upgrades static pods component-by-component with health gating.
    3. Updates the kube-proxy DaemonSet.
    4. Upgrades kubelet sequentially across nodes.
    5. Re-applies bootstrap manifests.

    Managing Drift and Configuration

    To ensure a smooth upgrade, you must coordinate talos_machine_configuration and talos_cluster:

    1. talos_machine_configuration: The kubernetes_version here controls the image tags embedded in the generated YAML. This is used for scale-up (new nodes bootstrap at this version).
    2. talos_cluster: The kubernetes_version here is the canonical driver for the actual upgrade procedure.

    Best Practice: To prevent talos_machine from bypassing the sequenced upgrade-k8s procedure by simply re-applying config with new image tags, set ignore_kubernetes_upgrade_drift = true on the talos_machine resource. This allows talos_cluster to fully own the upgrade lifecycle.

  10. Use the talos_client_configuration ephemeral resource

    main

    The talos_client_configuration resource is an ephemeral resource used to generate a Talos cluster client configuration from machine secrets.

    Key Characteristics:

    • Ephemeral: It does not persist secrets in the Terraform state.
    • Deterministic Output: The generated talos_config is byte-identical on every run as long as machine_secrets and not_before remain unchanged. This is achieved by pinning timestamps for the admin client certificate.

    To ensure stability and avoid unnecessary configuration changes, it is recommended to persist the not_before value in a terraform_data resource.

    # Example usage pattern
resource "talos_client_configuration" "example" {
  cluster_name = "my-cluster"
  not_before   = terraform_data.not_before.output
  machine_secrets = {
    # ... machine secrets configuration ...
  }
}

resource "terraform_data" "not_before" {
  output {
    value = timestamp()
  }
}

  11. Use the talos_cluster_health ephemeral resource

    main

    The talos_cluster_health resource is an ephemeral resource used to check the health of a Talos cluster. Because it is ephemeral, it does not persist secrets in your Terraform state. It validates the health of control plane and/or worker nodes by comparing the provided IPs against the addresses reported by Kubernetes.

    To use this resource, you must provide client_configuration containing the necessary certificates, a list of endpoints to reach the cluster, and at least one list of node IPs (control_plane_nodes or worker_nodes) to verify.

    # Example usage of talos_cluster_health

resource "talos_cluster_health" "example" {
  endpoints = ["192.168.1.10", "192.168.1.11"]

  client_configuration {
    ca_certificate     = "${file("ca.crt")}"
    client_certificate = "${file("client.crt")}"
    client_key         = "${file("client.key")}"
  }

  control_plane_nodes = ["192.168.1.10", "192.168.1.11"]
  
  # Optional: check worker nodes as well
  worker_nodes       = ["192.168.1.20"]

  # Optional: skip Kubernetes component checks (e.g., to just check if kubelet is running)
  skip_kubernetes_checks = false

  # Optional: set a custom timeout (e.g., 5 minutes)
  timeout = "5m"
}

  12. Upgrade multiple nodes safely

    main

    When upgrading a multi-node cluster (especially control planes), upgrading all nodes in parallel can cause loss of etcd quorum. To prevent this, use depends_on to chain upgrades sequentially so each node is fully back online before the next begins. Alternatively, you can run terraform apply -parallelism=1 to force sequential execution.

    resource "talos_machine" "cp0" {
  node             = "10.5.0.1"
  image            = "ghcr.io/siderolabs/installer:v1.10.0"
  drain_on_upgrade = true
  # ...
}

resource "talos_machine" "cp1" {
  depends_on       = [talos_machine.cp0]
  node             = "10.5.0.2"
  image            = "ghcr.io/siderolabs/installer:v1.10.0"
  drain_on_upgrade = true
  # ...
}