hcloud-talos Terraform Module

repository·main·Indexed 43 days ago

https://github.com/hcloud-talos/terraform-hcloud-talos

A Terraform module for deploying production-ready Kubernetes clusters on Hetzner Cloud using Talos OS. It features private networking, secure API exposure via firewall rules, and integration with Cilium CNI and the Hcloud Cloud Controller Manager. The module supports both x86 and ARM architectures and recommends using the terraform-provider-imager for creating Talos snapshots.

Tokens
6.3K
Snippets
14
Records
26
Agent score
56%

What's inside hcloud-talos

  1. Overview of Terraform - Hcloud - Talos

    main

    This Terraform module automates the creation of a Kubernetes cluster using the Talos OS on Hetzner Cloud.

    Key Features:

    • Production Ready: Designed following Talos production cluster recommendations.
    • Private Networking: Uses Hetzner Cloud Networks for internal node-to-node communication.
    • Secure API Exposure: Kubernetes and Talos APIs are secured via firewall rules. By default, firewall_use_current_ip = true restricts traffic to your current IP.
    • Customizable Networking: All network CIDRs (network, node, pod, service) can be customized.
    • Optimized Configuration: Includes manual network device configuration and Floating IP (VIP) provisioning.
  2. Build Talos Images for Hetzner Cloud (Legacy Packer Workflow)

    main

    This directory provides a legacy Packer-based workflow for building Talos OS images specifically for Hetzner Cloud.

    Warning: This workflow is deprecated and no longer maintained. For new clusters and ongoing updates, use hcloud-talos/imager instead. This directory is kept only as a fallback for existing users of the Packer flow.

    To build images, you can either use the default Talos images or provide custom image URLs from the Talos Image Factory to include specific system extensions.

  3. Configure Talos cluster endpoints and connectivity modes

    main

    The module supports several networking patterns for accessing the Kubernetes API and Talos API:

    VPN-only (Private bootstrap with WireGuard)

    Uses Talos's built-in WireGuard to manage the cluster over private IPs. Requires applying a WireguardConfig patch to the control plane and configuring a WireGuard client on your workstation.

    VPN-only (Site-to-site VPN gateway)

    Used when a VPN gateway VM exists on the same private network. Use cluster_api_host_private to point to the private VIP.

    Fully Private (No public IPv4)

    Nodes have no public IPs. All operations use private IPs via WireGuard or a VPN gateway. Set disable_public_ipv4 = true.

    Floating IP (Public VIP)

    Provides a stable public Kubernetes API endpoint without a dedicated load balancer. Set enable_floating_ip = true.

    External TCP Load Balancer

    Recommended for High Availability (HA). Points cluster_api_host to the Load Balancer DNS/IP and sets kubeconfig_endpoint_mode = "public_endpoint".

    Split-horizon

    Nodes use a private VIP/hostname, but the kubeconfig points to a public DNS/LB. Uses enable_alias_ip = true and cluster_api_host_private.

  4. Configure Public and Internal API Endpoints

    main

    The module allows you to define how the Kubernetes and Talos APIs are accessed.

    Public API Endpoint (Kubernetes API)

    Use the cluster_api_host variable to define a stable public hostname (e.g., kube.mydomain.com).

    • Requirement: You must create a DNS A record for this hostname pointing to the public IP (e.g., a Hetzner Floating IP or an external TCP load balancer).
    • Kubeconfig: If kubeconfig_endpoint_mode = "public_endpoint" is set, the generated kubeconfig will use this hostname.
    • Note: This controls the Kubernetes API (TCP/6443).

    Internal API Endpoint (Node-to-Node)

    Talos uses an internal hostname for communication between nodes. By default, this is kube.[cluster_domain] (e.g., kube.cluster.local).

    • Override: Use cluster_api_host_private to change this.
    • Alias IP: If enable_alias_ip = true (default), the module configures /etc/hosts on each node to resolve the internal hostname to a private alias IP (default 10.0.1.100).
    • Split-Horizon DNS: If accessing the cluster via VPN, consider setting cluster_api_host_private to a resolvable private name (e.g., kube.example.com -> 10.0.1.100) to prevent client-side DNS failures.

    Private Bootstrap

    If running Terraform from a host with VPN/private network access (e.g., WireGuard, Tailscale), set bootstrap_endpoint_mode = "private_ip". This allows Terraform to bootstrap and health-check the cluster via private IPs instead of public ones. This can be combined with disable_public_ipv4 = true to provision nodes without any public IPv4 addresses.

  5. Software installed in the cluster

    main

    The module automatically installs and configures several critical components:

    • Cilium: Used as the CNI (Container Network Interface) instead of Flannel. It provides Network Policies and Load Balancing. Note: cilium_version must be compatible with kubernetes_version.
    • Hcloud Cloud Controller Manager (CCM): Updates Node objects with Hetzner metadata, cleans up stale nodes, and manages type: LoadBalancer services by creating Hetzner Cloud Load Balancers.
    • Talos Cloud Controller Manager: Applies labels to nodes and validates/approves node Certificate Signing Requests (CSRs).
    • Tailscale (Optional): Provides secure, encrypted networking. Requirement: The Talos Image MUST be created with the tailscale extension when tailscale.enabled is set to true. Requires a valid Tailscale auth key.
  6. Run the extended Terraform example

    main

    This example demonstrates running the local module using terraform-provider-imager to create a Talos snapshot, which is then passed to the module via the talos_image_id_x86 variable.

    Prerequisites

    1. 1Password Configuration: Set the 1Password provider token using the environment variable TF_VAR_op_service_account_token_test. Alternatively, you can replace the 1Password configuration with direct hcloud and imager credentials.
    2. Variable Setup: Copy terraform.tfvars.example to terraform.tfvars and populate it with your specific vault and item UUIDs.

    Execution Steps

    1. Initialize the working directory:
      terraform init
    2. Apply the configuration. Note that the initial terraform apply may take several minutes because it must upload the Talos disk image to Hetzner Cloud and create a snapshot.
      terraform apply
    3. Export the generated credentials to local files:
      terraform output --raw kubeconfig > ./kubeconfig
      terraform output --raw talosconfig > ./talosconfig
    terraform init
    terraform apply
    terraform output --raw kubeconfig > ./kubeconfig
    terraform output --raw talosconfig > ./talosconfig
  7. Handoff cluster management to GitOps (Argo CD/Flux)

    main

    If you want to manage components like Cilium or the Hcloud CCM via GitOps tools instead of Terraform, you can hand off management after the initial bootstrap.

    1. Perform the initial terraform apply to bootstrap the cluster.
    2. Set deploy_cilium = false (and deploy_prometheus_operator_crds = false if used) in your Terraform configuration.
    3. Set deploy_hcloud_ccm = false if applicable.
    4. Run terraform apply again.

    Note: Because the module uses kubectl_manifest with apply_only = true, Terraform will remove these resources from its state without deleting the actual resources from the Kubernetes cluster.

  8. Build standard Talos images using create.sh

    main

    To build standard Talos images (both ARM and x86) without custom extensions, use the provided create.sh script.

    Prerequisites:

    • Packer must be installed.
    • A Hetzner Cloud API token must be available as the HCLOUD_TOKEN environment variable (the script will prompt you if it is missing).

    Execution: Run the following command in the _packer/ directory:

    ./create.sh

    What happens:

    1. The script validates the environment.
    2. It initializes Packer.
    3. It runs packer build using default image URLs defined in talos-hcloud.pkr.hcl.
    4. The resulting snapshots are stored directly in your Hetzner Cloud project.
    ./create.sh
  9. Export Talos and Kubernetes configuration files

    main

    To interact with your cluster using kubectl and talosctl, you must export the sensitive outputs from the Terraform module to local files.

    1. Define outputs in your Terraform configuration:
    output "talosconfig" {
      value     = module.talos.talosconfig
      sensitive = true
    }
    
    output "kubeconfig" {
      value     = module.talos.kubeconfig
      sensitive = true
    }
    1. Run the following commands to save the files:
    terraform output --raw kubeconfig > ./kubeconfig
    terraform output --raw talosconfig > ./talosconfig
  10. Create Talos Images with terraform-provider-imager

    main

    The recommended way to prepare Talos OS images for Hetzner Cloud is using the hcloud-talos/imager provider. This avoids issues with official Hetzner Talos ISOs, which may boot in metal mode instead of hcloud mode, causing incorrect providerIDs and networking failures.

    To use this workflow:

    1. Configure the imager provider with your hcloud_token.
    2. Define an imager_image resource using a Talos Factory URL.
    3. Pass the resulting snapshot IDs into the hcloud-talos module using talos_image_id_x86 and/or talos_image_id_arm.
    terraform {
      required_providers {
        imager = {
          source = "hcloud-talos/imager"
        }
      }
    }
    
    provider "imager" {
      token = var.hcloud_token
    }
    
    resource "imager_image" "talos_x86" {
      image_url    = "https://factory.talos.dev/image/<schematic-id>/<talos-version>/hcloud-amd64.raw.xz"
      architecture = "x86"
    
      labels = {
        version = var.talos_version
      }
    }
  11. Deploy a Talos cluster with Terraform

    main

    Deploy a Kubernetes cluster running Talos OS on Hetzner Cloud using the hcloud-talos/talos/hcloud module.

    Key Configuration Notes:

    • Version Compatibility: Ensure talos_version, kubernetes_version, and cilium_version are compatible with each other and the Talos Support Matrix.
    • Firewall Access: If firewall_use_current_ip is true, the module fetches your public IP via https://ipv4.icanhazip.com to allow API access. If false, you must manually provide IPs in firewall_kube_api_source and firewall_talos_api_source.
    • Node Definitions: Define control_plane_nodes and worker_nodes as lists of objects specifying id and type (e.g., cax11).
    module "talos" {
      source  = "hcloud-talos/talos/hcloud"
      version = "<latest-version>"
    
      talos_version      = "v1.12.2"
      kubernetes_version = "1.35.0"
      cilium_version     = "1.16.2"
    
      hcloud_token = "your-hcloud-token"
    
      cluster_name     = "dummy.com"
      cluster_domain   = "cluster.dummy.com.local"
      cluster_api_host = "kube.dummy.com"
    
      firewall_use_current_ip  = false
      firewall_kube_api_source = ["your-ip"]
      firewall_talos_api_source = ["your-ip"]
    
      location_name = "fsn1"
    
      control_plane_nodes = [
        { id = 1, type = "cax11" },
        { id = 2, type = "cax11" },
        { id = 3, type = "cax11" }
      ]
      control_plane_allow_schedule = true
    
      worker_nodes = [
        { id = 1, type = "cax21" },
        { id = 2, type = "cax21" },
        { id = 3, type = "cax21" }
      ]
    
      network_ipv4_cidr = "10.0.0.0/16"
      node_ipv4_cidr    = "10.0.1.0/24"
      pod_ipv4_cidr     = "10.0.16.0/20"
      service_ipv4_cidr = "10.0.8.0/21"
      
      tailscale = {
        enabled  = true
        auth_key = "tskey-auth-xxxxxxxxxxxx"
      }
    }