DOCSEARCHbyAlgolia
Submit Docs

k0s Kubernetes Distribution

repository·main·Indexed 11 days ago

https://github.com/k0sproject/k0s

A zero-friction, all-inclusive, and CNCF-certified Kubernetes distribution delivered as a single binary. Designed to be lightweight with minimum requirements of 1 vCPU and 1 GB RAM, k0s is suitable for cloud, bare metal, edge, and IoT environments. It supports single-node, multi-node, and airgapped installations, and can be deployed via k0sctl or within Docker containers.

Tokens
108K
Snippets
395
Records
500
Agent score
90%

What's inside k0s

  1. Overview of k0s Kubernetes distribution

    main

    k0s is an all-inclusive, open-source Kubernetes distribution packaged as a single binary. It is designed for zero friction, requiring no host OS dependencies other than the kernel. This makes it suitable for cloud, bare metal, edge, and IoT environments.

    Key characteristics include:

    • Single Binary: Easy to distribute and deploy with no external runtime dependencies.
    • Lightweight: Minimum system requirements are 1 vCPU and 1 GB RAM.
    • Upstream Compliance: 100% upstream, CNCF-certified Kubernetes.
    • Flexible Deployment: Supports single-node, multi-node, airgap, and Docker-based installations.
    • Extensible: Supports custom CNI, CRI, and CSI plugins.

  2. Overview of Certificate Authorities (CAs) in k0s

    main

    k0s manages three distinct cryptographic entities to secure the cluster:

    • Kubernetes CA: Secures the Kubernetes cluster and manages client/server certificates for API communication.
    • etcd CA: Used exclusively when managed etcd is enabled to secure etcd communications.
    • Kubernetes Service Account (SA) key pair: Used for signing Kubernetes service account tokens.

    These CAs are automatically generated during cluster initialization with a default expiration of 10 years. They are distributed to all k0s controllers during the join process. Note that k0s does not currently provide automated CA renewal; replacing them is a manual process.

  3. What is Control Plane Load Balancing (CPLB)?

    main

    Control Plane Load Balancing (CPLB) is a feature used to provide a highly available Virtual IP (VIP) and load balancing for accessing a k0s cluster externally when an externally managed load balancer is unavailable.

    Key characteristics:

    • VIP (Virtual IP): An IP address that floats between multiple control plane nodes, ensuring high availability and removing single points of failure.
    • Load Balancing: Forwards traffic to every control plane node.
    • Compatibility with NLLB: CPLB is fully compatible with Node Local Load Balancing (NLLB), which handles internal (node-to-control-plane) traffic. It is recommended to use both together if you lack an external load balancer.
    • Mechanism: CPLB relies on keepalived and the VRRP protocol.

  4. What is node-local load balancing in k0s?

    main

    Node-local load balancing is a mechanism used to achieve internal high availability for the k0s control plane when an externally managed load balancer is not available.

    Unlike an external load balancer, node-local load balancing operates exclusively on the worker nodes. It manages a load balancer on each worker node's loopback interface, allowing worker components to distribute requests among all available controller nodes.

    Key characteristics:

    • Internal Resilience: It makes the cluster resilient to controller node outages from within the cluster, but it does not make the control plane highly available to external clients (e.g., kubectl or Lens users interacting from outside the cluster).
    • Backing Load Balancers: k0s supports Envoy (the default) and Traefik.
    • Platform Constraints: Envoy is not available for ARMv7, RISC-V, or Windows. If running on these platforms, you must use Traefik.

  5. Apply component patches via `spec.network`

    main

    Experimental Feature: Component Patching

    As an experimental feature, patching allows you to customize Kubernetes resources generated by k0s components before they are applied. This is useful for adjustments not exposed by standard configuration options.

    Supported Components and Keys:

    • CoreDNS: spec.network.coreDNS.patches
    • kube-proxy: spec.network.kubeProxy.patches
    • kube-router: spec.network.kuberouter.patches
    • Calico: spec.network.calico.patches
    • metrics-server: spec.metricsServer.patches

    Patch Structure: Each patch entry requires:

    • target.kind: The Kubernetes kind (e.g., Deployment, Service).
    • target.name: The metadata.name of the resource.
    • target.namespace: (Optional) The namespace of the resource.
    • patch.type: One of JSON (RFC 6902), MergePatch (RFC 7386), or StrategicMergePatch (Note: StrategicMergePatch is only supported for built-in Kubernetes resource kinds).
    • patch.content: The patch body in JSON or YAML format.

    Multiple patches matching the same resource are applied in the order they are listed.

    spec:
  network:
    coreDNS:
      patches:
        - target:
            kind: Deployment
            name: coredns
            namespace: kube-system
          patch:
            type: StrategicMergePatch
            content: |
              spec:
                replicas: 3
                template:
                  spec:
                    containers:
                      - name: coredns
                        resources:
                          limits:
                            memory: 256Mi

  6. Backup and Restore Overview

    main

    k0s provides integrated support for backing up and restoring cluster state and configuration. These operations must be performed on the controller node.

    What is included in a backup:

    • Certificates (content of <data-dir>/pki)
    • etcd snapshot (if using etcd)
    • Kine/SQLite snapshot (if using Kine/SQLite)
    • k0s.yaml configuration
    • Custom manifests under <data-dir>/manifests
    • Image bundles under <data-dir>/images
    • Helm configuration

    What is NOT included:

    • PersistentVolumes of running applications
    • Data stores other than etcd or Kine/SQLite
    • Manual cluster configuration changes not saved under <data-dir>/manifests

  7. Understand Kubernetes service exposure options

    main

    When exposing services to external networks in k0s, you have three primary options:

    1. NodePort: Configures a port on a node to route traffic to a service. The port range is limited to 30000-32767, meaning you cannot use standard ports like 80 or 443.
    2. LoadBalancer: Typically provided by cloud providers or implemented internally (e.g., using MetalLB for bare-metal). It provides a single IP address to access services running on multiple nodes.
    3. Ingress Controller: Consolidates routing rules for multiple applications into one entity. It can be exposed via NodePort, LoadBalancer, or the host network. It also provides a centralized location for TLS termination.

  8. How the k0s Control Plane works

    main

    The k0s binary acts as a process supervisor for all control plane components. Instead of running components inside a container engine, k0s runs them as "naked" processes directly on the host.

    Important Constraint: Because there is no container engine or kubelet running on controller nodes by default, you cannot schedule user workloads onto controller nodes. Controller nodes are dedicated to managing the cluster components.

  9. Understand the k0s target use cases and personas

    main

    k0s is a streamlined, self-contained Kubernetes distribution designed as a portable and lightweight foundational core. It is specifically optimized for:

    • Edge Computing and IoT: Managing resource-constrained, disconnected, or air-gapped environments. k0s supports air-gapped setups by automatically loading container image bundles and having zero dependencies.
    • Cloud-Native Application Hosting: Running containerized applications with minimal operational complexity.
    • Dev/Test Environments: Quickly spinning up Kubernetes clusters locally or in isolated CI/CD pipelines.
    • Centralized Control for Distributed Clusters: Managing control planes centrally while operating worker nodes in remote locations. k0s uses konnectivity by default, making it ideal for isolated control planes.
    • Heterogeneous Environments: Running on bare-metal or hybrid cloud/on-premises setups without heavy reliance on external cloud services.

    Target Personas:

    • DevOps & Platform Engineers: Seeking automation and integration into CI/CD and monitoring.
    • IoT & Edge Architects: Requiring resource-efficient solutions for the far edge.
    • Cloud-Native Developers: Needing minimal setup and high portability.
    • System Integrators & IT Ops: Designing bespoke or large-scale multi-cluster operations.

  10. What happens during a k0s reset

    main

    A k0s reset is designed to remove all components, configurations, and data associated with k0s from a node. The following actions are performed:

    • Processes and containers: Terminates all running k0s processes and container processes managed by the Container Runtime.
    • Mounts: Unmounts all mount points under the k0s data directory (using lazy unmount if necessary).
    • Data deletion: Deletes the entire k0s data directory, including:
      • Configuration files (cluster setup and node-specific settings).
      • Managed data stores (etcd, kine, etc.).
      • Certificates and key material used for authentication.
    • Network settings: Reverts network interfaces and iptables rules on a best-effort basis. (Rebooting is recommended to ensure complete cleanup).
    • Init system: Removes the registration created by k0s install, so k0s will no longer start automatically on boot.

  11. Identify controller-node specific configuration options

    main

    While most cluster-wide settings are managed dynamically via the Kubernetes API, certain options are node-specific and must be configured via the local configuration file on every controller node. These include:

    • spec.api: Configures the local Kubernetes API server setup.
    • spec.storage: Configures local storage (etcd or SQLite) setup.
    • spec.network.controlPlaneLoadBalancing: Configures Control Plane Load Balancing (CPLB).

    In High Availability (HA) control plane setups, all controllers must have these local configurations defined to ensure storage and the API server can run correctly.

  12. Autopilot Plan and Command Processing States

    main

    Autopilot manages the lifecycle of Plans and Commands through a specific set of states. Understanding these states is critical for debugging why an update might be stalled or failed.

    Standard States

    • NewPlan: Triggered when a Plan named autopilot is created. In this state, the system iterates over all commands to ensure their status entries are initialized in the Plan status.
    • SchedulableWait: An evaluation state used to determine if a command is ready to be processed. It acts as a buffer between scheduling and execution.
    • Schedulable: Indicates the command is ready for execution. The logic defined by the specific command (e.g., k0supdate) is performed in this state.
    • Completed: The command or plan has finished successfully. No further processing occurs once this state is reached.

    Error States (Fatal)

    If a command enters an error state, the plan aborts. Common errors include:

    • IncompleteTargets: Occurs during airgapupdate or k0supdate if a Signal Node discovered during the NewPlan phase no longer exists (e.g., the Node object was deleted).
    • Restricted: Occurs if a Plan requests an update for a node type that was explicitly excluded using the --exclude-from-plans argument during startup.