DOCSEARCHbyAlgolia
Submit Docs

hetzner-k3s Documentation

repository·main·Indexed 25 days ago

https://github.com/vitobotta/hetzner-k3s

A CLI tool for rapidly deploying production-ready Kubernetes clusters on Hetzner Cloud. It automates the setup of masters, workers, networking, and essential controllers—including the Hetzner Cloud Controller Manager, CSI Driver, and Cluster Autoscaler—without requiring Terraform or manual management clusters. Supports high availability with multi-location masters, custom node pools, and automated scaling.

Tokens
23.2K
Snippets
66
Records
116
Agent score
87%

What's inside hetzner-k3s

  1. Overview of hetzner-k3s

    main

    hetzner-k3s is a CLI tool designed to rapidly provision production-ready Kubernetes clusters on Hetzner Cloud. It uses the lightweight k3s distribution and automates the configuration of essential production components.

    Key features include:

    • Automated Provisioning: Sets up k3s, Hetzner Cloud Controller Manager (for load balancers), Hetzner CSI Driver (for block storage), System Upgrade Controller, and Cluster Autoscaler.
    • Infrastructure as Code: Uses simple, human-readable YAML configuration.
    • Security: Configures private networking and firewalls automatically to keep cluster traffic off the public internet.
    • High Availability: Supports distributed master and worker nodes across different locations.
    • Zero Lock-in: No management cluster required (unlike Cluster API) and no third-party access to your credentials.

  2. Understand the test execution flow

    main

    Each test follows this lifecycle:

    1. Executes hetzner-k3s create.
    2. Executes basic kubectl commands.
    3. Executes hetzner-k3s delete.

    Tests are executed in separate directories named test-xxxxxxxx. Logs for each phase are stored within these directories, along with status files used to track success or failure.

  3. Compare hetzner-k3s with alternative Kubernetes deployment methods

    main

    Use the following comparison to determine if hetzner-k3s fits your requirements compared to Managed Services, Terraform-based solutions, Cluster API, or Claudie.

    Factorhetzner-k3sManaged ServicesTerraform-basedCluster API
    Setup time2-3 minutes5-10 minutes15-30+ minutes20+ minutes
    DependenciesCLI onlyAccount signupTerraform, Packer, HCLManagement cluster
    Data privacyFull controlThird-party accessFull controlFull control
    Monthly costInfrastructure onlyInfrastructure + platform feesInfrastructure onlyInfrastructure only
    Credential exposureNoneAPI tokens to third partyNoneNone
    Learning curveLowLowMedium-HighHigh
    Best forMost Hetzner usersZero-ops teamsTerraform-native teamsMulti-cloud standardization

  4. Getting Started with hetzner-k3s

    main

    To begin using hetzner-k3s, follow these primary steps:

    1. Installation: Install the CLI tool on your local system (supports macOS, Linux, or Windows via WSL).
    2. Configuration: Define your cluster requirements using a YAML configuration file.
    3. Cluster Creation: Run the CLI to provision your infrastructure and Kubernetes cluster.
    4. Stack Setup: (Optional) Configure ingress, TLS, and deploy sample applications to complete your environment.

  6. Re-run a cached test

    main
    The test harness uses a caching system: running the same configuration and parameters twice will skip the test. To force a re-run of a specific test, delete its corresponding directory (e.g., test-xxxxxxxx). You can find the specific test directory name by running ./list-test-results.sh.

  7. Recreate a master in the correct location

    main

    If a master is deployed in the wrong location during a regional cluster conversion, follow these steps to force it into the correct location (e.g., for master1 to ensure it is in fsn1):

    1. Drain the node: kubectl drain {cluster-name}-master1.
    2. Delete the node from Kubernetes: kubectl delete node {cluster-name}-master1.
    3. Remove the instance via the Hetzner Console or the hcloud utility.
    4. Run the create command again. This will recreate the master in the correct location based on the alphabetical order of the locations list.

    Repeat these steps for master2 (to ensure hel1) or master3 (to ensure nbg1) if necessary.

  8. Configure a Small to Medium Cluster (1-50 nodes)

    main

    For clusters up to 50 nodes, use the default configuration which relies on Hetzner Private Networks for security and simplicity.

    Key Recommendations:

    • High Availability: Use 3 master nodes.
    • Networking: Enable private_network and use flannel CNI for simplicity.
    • Storage: Use hcloud-volumes for persistent data.
    • Load Balancers: Use Hetzner Load Balancers for production workloads.
    hetzner_token: <your token>
cluster_name: my-cluster
kubeconfig_path: "./kubeconfig"
k3s_version: v1.32.0+k3s1

networking:
  ssh:
    port: 22
    use_agent: false
    public_key_path: "~/.ssh/id_ed25519.pub"
    private_key_path: "~/.ssh/id_ed25519"
  allowed_networks:
    ssh:
      - 0.0.0.0/0
    api:
      - 10.0.0.0/16  # Restrict to private network
  public_network:
    ipv4: true
    ipv6: true
  private_network:
    enabled: true
    subnet: 10.0.0.0/16
  cni:
    enabled: true
    encryption: false
    mode: flannel

masters_pool:
  instance_type: cpx22
  instance_count: 3  # For HA
  locations:
    - nbg1

worker_node_pools:
- name: workers
  instance_type: cpx32
  instance_count: 3
  location: nbg1
  autoscaling:
    enabled: true
    min_instances: 1
    max_instances: 5

protect_against_deletion: true
create_load_balancer_for_the_kubernetes_api: true

  9. Run end-to-end tests

    main

    The test harness provides scripts to run tests individually or in bulk.

    Run a single test

    Execute ./run-single-test.sh followed by a configuration file template and optional variable substitutions.

    Run all tests

    Execute ./run-all-tests.sh to run the entire suite sequentially.

    View test results

    Execute ./list-test-results.sh to see a summary of test statuses, configurations, and test IDs.

  10. Configure a Large Cluster (50+ nodes)

    main

    Hetzner's private networks are limited to 100 nodes. For clusters exceeding this, you must disable the private network and use the public network with Wireguard encryption.

    Architecture Requirements:

    • IP Query Server: Required to manage custom firewalls and node IPs.
    • Networking: Disable private_network, enable cni.encryption, and use cilium CNI.
    • CIDR: Use larger cluster_cidr and service_cidr ranges.
    • Datastore: Consider using an external PostgreSQL datastore for very large scales.
    • Firewall: Set use_local_firewall: true to use the custom firewall solution.
    hetzner_token: <your token>
cluster_name: large-cluster
kubeconfig_path: "./kubeconfig"
k3s_version: v1.32.0+k3s1

networking:
  ssh:
    port: 22
    use_agent: true
    public_key_path: "~/.ssh/id_ed25519.pub"
    private_key_path: "~/.ssh/id_ed25519"
  allowed_networks:
    ssh:
      - 0.0.0.0/0
    api:
      - 0.0.0.0/0
  public_network:
    ipv4: true
    ipv6: true
    hetzner_ips_query_server_url: https://ip-query.example.com
    use_local_firewall: true
  private_network:
    enabled: false
  cni:
    enabled: true
    encryption: true
    mode: cilium

cluster_cidr: 10.244.0.0/15
service_cidr: 10.96.0.0/16
cluster_dns: 10.96.0.10

datastore:
  mode: etcd
  # external_datastore_endpoint: postgres://...

masters_pool:
  instance_type: cpx32
  instance_count: 3
  locations:
    - nbg1
    - hel1
    - fsn1

worker_node_pools:
- name: compute
  instance_type: cpx42
  location: nbg1
  autoscaling:
    enabled: true
    min_instances: 5
    max_instances: 50
- name: storage
  instance_type: cpx52
  location: hel1
  autoscaling:
    enabled: true
    min_instances: 3
    max_instances: 20

addons:
  embedded_registry_mirror:
    enabled: true

protect_against_deletion: true
create_load_balancer_for_the_kubernetes_api: true
k3s_upgrade_concurrency: 2

  11. Prerequisites for setting up a Hetzner k3s cluster

    main

    Before using hetzner-k3s to create a cluster, ensure you have the following ready:

    1. Hetzner Cloud Account: Must have a project and an API token.
    2. SSH Key Pair: Required for accessing cluster nodes.
    3. kubectl: Installed on your local machine.
    4. Helm: Installed on your local machine.
    5. hetzner-k3s: Installed on your local machine.

  12. Scale down a node pool

    main

    To reduce the size of a node pool, follow these steps:

    1. Lower the instance_count in your configuration file to prevent the extra nodes from being recreated.
    2. Drain and delete the additional nodes from Kubernetes. Typically, these are the nodes with the highest alphabetical names.
    3. If the Cloud Controller Manager does not automatically remove the instances, manually delete them from the Hetzner Cloud console.