OpenStack-Helm

repository·master·Indexed 20 days ago

https://github.com/openstack/openstack-helm

A collection of Helm charts designed to deploy OpenStack services on Kubernetes. It includes specialized components such as the ceph-adapter-rook for interfacing with Rook-managed Ceph clusters, the deploy-env Ansible role for automating Kubernetes test environment setup, and various service-specific templates for Aodh, Barbican, Blazar, and Ceilometer including database synchronization and bootstrapping scripts.

Tokens
137.4K
Snippets
396
Records
533
Agent score
69%

What's inside OpenStack-Helm

  1. Overview of OpenStack-Helm

    master
    OpenStack-Helm is a collection of Helm charts designed to deploy OpenStack and related services on Kubernetes. It aims to provide a simple, resilient, and flexible deployment mechanism for OpenStack services within a Kubernetes environment.
  2. Overview of Rally benchmarking tool

    master
    Rally is a benchmarking tool for OpenStack services used to test cloud performance at scale. This Helm chart leverages the Kolla image for Rally and provides a templated configuration file that supports configuration overrides, consistent with other OpenStack-Helm charts. You can control which OpenStack services are benchmarked by modifying the enabled_tests key in the values.yaml file.
  3. Deploy a MariaDB Galera Cluster with openstack-helm

    master

    The MariaDB chart deploys a 3-member Galera cluster using Kubernetes StatefulSets with persistent storage.

    Deployment Lifecycle:

    1. Seed Job: A temporary standalone Galera cluster is created via a Kubernetes Job. This job bootstraps authentication and exposes WSREP bindings.
    2. StatefulSet Provisioning: Cluster members are provisioned one at a time. A member must pass its readinessProbes (validating MySQL is up and responsive) before the next member is provisioned.
    3. Synchronization: The cluster uses xtrabackup-v2 for synchronization.
    4. Cluster Completion: The seed job completes only when Galera reports as Synced and the number of reporting cluster members matches the replica count defined in your Helm values.
    5. Steady State: Once the seed job terminates, new StatefulSet members will use the existing cluster members as gcomm endpoints instead of the seed job. This mechanism allows for scaling and restarting members.

    Storage: StatefulSets use Persistent Volume Claims (PVCs) to provide stateful storage for /var/lib/mysql.

  4. Configure Ansible inventory groups for deploy-env

    master

    The deploy-env role relies entirely on specific Ansible inventory groups to determine where components are installed. You must define the following groups:

    • primary: The deployer's machine. This is where kubectl and helm CLI tools are installed. This group must contain exactly one node.
    • k8s_control_plane: The node(s) where the Kubernetes control plane is deployed. This group must contain exactly one node.
    • k8s_cluster: All nodes belonging to the Kubernetes cluster, including both control plane and worker nodes.
    • k8s_nodes: The worker nodes in the cluster.

    Special Configurations:

    • Single-node setup: If you want the control plane node to also run all workloads (single-node cluster), the k8s_nodes group must be empty. In this case, the primary node and the k8s_control_plane node can be the same physical machine.
    • Multi-node setup: Include your worker nodes in the k8s_nodes group.
    all:
      vars:
        ansible_port: 22
        ansible_user: ubuntu
        ansible_ssh_private_key_file: /home/ubuntu/.ssh/id_rsa
        ansible_ssh_extra_args: -o StrictHostKeyChecking=no
      hosts:
        primary:
          ansible_host: 10.10.10.10
        node-1:
          ansible_host: 10.10.10.11
        node-2:
          ansible_host: 10.10.10.12
        node-3:
          ansible_host: 10.10.10.13
      children:
        primary:
          hosts:
            primary:
        k8s_cluster:
          hosts:
            node-1:
            node-2:
            node-3:
        k8s_control_plane:
          hosts:
            node-1:
        k8s_nodes:
          hosts:
            node-2:
            node-3:
  5. Manage index lifecycle with Elastic Curator

    master

    The Elasticsearch chart includes a CronJob for Elastic Curator to automate index management tasks such as taking snapshots, deleting old indices, restoring indices, or reindexing.

    Important Configuration Rules:

    • Configuration is handled via the conf.curator section in values.yaml.
    • Do not use "None" for unused fields. If a field is not needed, leave it blank. Entering the string "None" will cause Curator to interpret it as a Python NoneType, resulting in an exception.

    Curator configuration is split into two main parts: client (connection settings) and action_file (the actual lifecycle tasks).

    conf:
      curator:
        client:
          hosts: ["elasticsearch-logging"]
          port: 9200
          use_ssl: False
          # Leave unused fields blank, not "None"
          certificate: 
        logging:
          loglevel: INFO
        action_file:
          actions:
            1:
              action: delete_indices
              description: "Clean up ES by deleting old indices"
              options:
                disable_action: True # Actions are disabled by default
              filters:
                - filtertype: age
                  source: name
                  direction: older
                  timestring: '%Y.%m.%d'
                  unit: days
                  unit_count: 30
  6. Configure services using values-driven config files

    master

    To maximize flexibility and facilitate upgrades without modifying the Helm charts themselves, OpenStack-Helm uses values-driven configuration.

    Configuration files are specified dynamically based on the values.yaml file and any provided overrides. There are two primary methods for defining configuration:

    1. Dynamic Generation: Config files are automatically generated based on the structure of the YAML values tree.
    2. String Literals: The exact content of a configuration file can be specified directly as a string literal within the values.yaml file.
  7. When to draft a project specification

    master

    A specification is required before implementing broad-reaching technical changes or proposals in OpenStack-Helm. Use a specification when proposing changes such as:

    • Defining a standard format for values.yaml files.
    • Adding multiple backend support for Neutron.
    • Defining the approach for logging and monitoring within OpenStack-Helm.

    Note that introducing a new service may not require a new specification if it falls under the scope of an existing, approved specification.

  8. Understand OpenStack-Helm chart versioning and compatibility

    master

    OpenStack-Helm charts follow a specific versioning scheme that is distinct from the OpenStack release version.

    Key Concepts:

    • Chart Version vs. OpenStack Release: The chart version (e.g., 2024.2.X) does not equal the OpenStack release version. The OpenStack version is defined by the container images used with the charts.
    • Compatibility: A chart version like 2024.2.X is designed to be compatible with the OpenStack releases maintained at the time it was built (typically the 3 most recent versions, e.g., 2023.1, 2023.2, 2024.1, and 2024.2).
    • Version Format: Tarball versions follow the pattern X.Y.Z+<osh_commit_sha>_<osh_infra_commit_sha>. The X.Y.Z part represents the OpenStack release version, and the suffix provides build metadata for traceability.
  9. How to integrate an SDN using OpenVSwitch (OVS)

    master

    If your custom SDN uses OpenVSwitch as its network virtualization engine, you can leverage the existing OVS daemonsets provided by the base Neutron chart.

    To do this, you should:

    1. Disable the default Neutron L2 agent (daemonset_ovs_agent: false).
    2. Keep the OVS tools enabled (daemonset_ovs_db: true and daemonset_ovs_vswitchd: true).
    3. Provide the necessary SDN plugin code (either via a modified neutron-server image or by mounting the plugin into the original image).
    4. Configure neutron.conf and ml2_conf.ini to use the new plugin.
    manifests:
      # Neutron L2 agent: disable default
      daemonset_ovs_agent: false
      # OVS tool: keep enabled
      daemonset_ovs_db: true
      daemonset_ovs_vswitchd: true
  10. Understand the OpenStack-Helm Monitoring Stack (Prometheus/Grafana)

    master

    OpenStack-Helm utilizes a Prometheus and Grafana-based stack for metrics collection, alerting, and visualization.

    Core Components:

    • Prometheus: Scrapes metrics from targets via HTTP and stores them in a time-series database. It uses a specific querying language and supports service discovery or static configuration. It is used to evaluate alerting rules.
    • Alertmanager: Interfaces with Prometheus to handle and send alerts based on rule evaluations.
    • Grafana: Connects to Prometheus as a data source to provide single-pane dashboards. Dashboards can be bootstrapped via ConfigMaps or added via the web interface.
    • Exporters: Specialized components (e.g., kube-state-metrics, ceph-exporter, openstack-exporter) that translate service-specific metrics into a format Prometheus can scrape.

    Monitored Metrics Categories:

    • Host Utilization: CPU, memory, disk I/O, and network I/O.
    • Kubernetes: Pod status, replica availability, and job status.
    • Ceph: Pool usage, latency, and health.
    • OpenStack: Tenants, networks, flavors, floating IPs, and quotas.
  11. How Neutron agents and services are scheduled

    master

    In OpenStack-Helm, Neutron services are scheduled based on specific node labels:

    • neutron-server, neutron-rpc-server, neutron-dhcp-agent, neutron-l3-agent, and neutron-metadata-agent: These are scheduled on nodes with the label openstack-control-plane=enabled.
    • neutron-ovs-agent: Runs as a daemonset on nodes labeled openvswitch=enabled (which includes both compute and controller/network nodes).
    • OpenVSwitch (ovs-vswitchd/db): Runs on nodes labeled openvswitch=enabled.