DOCSEARCHbyAlgolia
Submit Docs

AXIVO K3s Cluster

repository·main·Indexed 43 days ago

https://github.com/axivo/k3s-cluster

An Ansible-driven automation framework for deploying high-availability K3s Kubernetes clusters on bare-metal or VMs. It provides a production-ready stack featuring Cilium CNI (eBPF), Longhorn distributed storage, ArgoCD for GitOps, and a full observability suite including VictoriaMetrics, VictoriaLogs, and Grafana. Designed specifically for Ubuntu LTS environments, it automates the entire cluster lifecycle from deployment and configuration to maintenance and upgrades.

Tokens
25K
Snippets
18
Records
73
Agent score
54%

What's inside AXIVO K3s Cluster

  2. Overview of the argo-cd role

    main

    The argo-cd role manages the deployment, reset, and validation of Argo CD within the cluster. It utilizes the official Argo CD Helm chart for deployment.

    Key Dependencies:

    • Repository: https://github.com/argoproj/argo-cd
    • Name: argocd
    • Version: 3.3.7

    Best Practices:

    • Use Renovate to automate release pull requests and keep dependencies up-to-date.
    • Use Robusta KRR to optimize cluster resource allocation.

  3. Overview of AXIVO K3s Cluster

    main
    AXIVO K3s Cluster is an Ansible-driven framework designed to deploy and manage production-ready Kubernetes clusters on bare-metal or virtual machines. It automates the entire cluster lifecycle, including deployment, configuration, maintenance, and upgrades, specifically targeting Ubuntu LTS environments.

  5. Overview of the cert-manager role

    main

    The cert-manager role manages the deployment, reset, and validation of the cert-manager Helm chart within a K3s cluster. It handles the installation of the cert-manager components and manages the cmctl CLI dependency.

    Key Dependencies:

    • cmctl (version 2.4.1) from https://github.com/cert-manager/cmctl.

  6. Key features of the AXIVO K3s Cluster

    main

    The cluster provides a production-grade stack with the following capabilities:

    • High Availability (HA): K3s with embedded etcd, using HAProxy and Keepalived for Kubernetes API server load balancing.
    • Networking: Cilium CNI (eBPF-based) for optimized service load balancing (replacing kube-proxy) and Gateway API for ingress.
    • DNS: CoreDNS for internal resolution and ExternalDNS for external record synchronization.
    • Security: TLS management via cert-manager with automatic renewal.
    • Storage: Distributed block storage using Longhorn for volume replication and backup.
    • Observability: A full stack including Metrics Server, VictoriaMetrics (metrics), AlertManager (alerts), VictoriaLogs (logging), and Grafana (visualization).
    • GitOps: ArgoCD for declarative application deployment.
    • Maintenance: Kured for coordinated node updates to minimize downtime.

  7. Configure Ansible roles using the standard pattern

    main

    All roles in this repository follow a consistent configuration and structure pattern. When creating or modifying roles, adhere to these conventions:

    Variable Naming

    • Use a specific namespace for variables in defaults/main.yaml using the <role>_vars prefix (e.g., cilium_vars).
    • Use <role>_map and <role>_project for facts and computed values.

    Role Structure

    • defaults/main.yaml: Default variables.
    • tasks/main.yaml: Main entry point.
    • tasks/validation.yaml: Prerequisite checks.
    • tasks/facts.yaml: Fact gathering.
    • tasks/postinstall.yaml: Post-deployment tasks.
    • tasks/upgrade.yaml: Version update procedures.
    • tasks/reset.yaml: Cleanup/removal procedures.
    • templates/: Jinja2 templates for values.yaml or Kubernetes manifests.
    • handlers/: Service management and restarts.

    Execution Patterns

    • Cluster-wide operations: Use run_once: true (typically on the first server).
    • Node-specific execution: Use conditionals like when: inventory_hostname in k3s_map.server.hosts.
    • Error Handling: Use any_errors_fatal: true for validation checks and implement retry logic with delay, retries, and until for network operations.

  8. Manage Helm plugins

    main

    The helm role supports the installation of plugins. By default, the diff plugin is included. You can manage plugins through the helm_vars.plugins collection.

    Default plugin configuration:

    • Name: diff
    • Version: v3.15.6
    • Repository: https://github.com/databus23/helm-diff
    • Dependencies: python3-jsonpatch
    # Example of enabling/disabling the diff plugin
helm_vars:
  plugins:
    - name: "diff"
      enabled: true
      version: "v3.15.6"
      repository:
        url: "https://github.com/databus23/helm-diff"

  9. Configure cluster role variables

    main

    The cluster role is configured using a set of variables prefixed with cluster_vars. Default values are located in defaults/main.yaml, while advanced variables are managed via the cluster_map collection in tasks/facts.yaml.

    Key configuration areas include:

    • Hardware & Device: Define architecture, product type, and specific storage device IDs.
    • Services: Enable or disable services like bluetooth, cloud_init, snapd, wifi, and postfix.
    • Security & Maintenance: Configure unattended_upgrades and SSH key paths.
    • Postfix/Mail: Configure SMTP relay hosts, ports, and protocols for mail services.

  10. Configure CoreDNS deployment via role variables

    main

    The coredns role manages the deployment, reset, and validation of CoreDNS using Helm. You can customize the deployment by providing specific variables.

    Key configuration areas include:

    • Helm Settings: Control the chart name, version, and repository URL.
    • HPA (Horizontal Pod Autoscaler): Enable or disable autoscaling and define replica limits and resource utilization targets.
    • Resource Management: Set CPU and memory requests and limits.
    • Namespace: Define which Kubernetes namespace CoreDNS should be deployed into (defaults to kube-system).

    For advanced configuration, refer to the coredns_map collection in tasks/facts.yaml.

  11. Understand the K3s Cluster Provisioning Workflow

    main

    The K3s cluster is provisioned using Ansible roles applied in a specific sequence to ensure dependencies are met. The workflow follows this order:

    1. Base System: cluster (Hardware and OS optimization)
    2. Kubernetes: k3s (Core K3s installation)
    3. Package Manager: helm (Helm CLI and repos)
    4. Networking: cilium $\rightarrow$ coredns $\rightarrow$ external-dns (CNI and DNS)
    5. Security: cert-manager (TLS management)
    6. Storage: longhorn (Distributed block storage)
    7. Observability: metrics-server $\rightarrow$ victoria-logs $\rightarrow$ victoria-metrics (Monitoring)
    8. Application Management: argo-cd $\rightarrow$ kured (GitOps and reboots)

  12. Configure VictoriaMetrics K8s Stack variables

    main

    The victoria-metrics-k8s-stack role uses several variables for configuration. Default values are located in defaults/main.yaml. Advanced users can access the victoriametricsk8sstack_map collection defined in tasks/facts.yaml.

    Best Practices

    • Use Renovate to automate release pull requests and keep dependencies up-to-date.
    • Use Robusta KRR to optimize cluster resource allocation.