DOCSEARCHbyAlgolia
Submit Docs

Hyperconverged Cluster Operator

repository·main·Indexed 41 days ago

https://github.com/kubevirt/hyperconverged-cluster-operator

A unified management layer for KubeVirt and its ecosystem that automates the deployment and lifecycle of KubeVirt, Containerized Data Importer (CDI), and other helper operators including the Scheduling, Scale and Performance (SSP) Operator, Cluster Network Addons Operator, and Node Maintenance Operator.

Tokens
56.2K
Snippets
153
Records
214
Agent score
51%

What's inside HCO

  1. Overview of CI Test Suites

    main

    The HCO project uses several CI test layers to ensure stability:

    Pre-submit Tests

    Run on every push to a PR and after merging.

    • kubevirt-ci: Deploys to a plain Kubernetes cluster to run functional tests, check object labels, verify default values, and ensure the HyperConverged CR can be deleted.
    • openshift-ci: Runs on OCP/OKD clusters (AWS, Azure, or GCP). Includes upgrade tests (installing latest main, then upgrading to PR version) and upgrade-prev tests (upgrading from the previous version branch). It verifies OVS annotations, labels, and namespace deletion constraints.

    Post-submit Tests

    Runs after merging a PR to the required branch (e.g., pull-hyperconverged-cluster-operator-e2e-k8s-1.19).

    Periodic Tests

    Nightly tests that build and deploy the latest version from the main branch alongside the latest KubeVirt version.

  2. What is the Hyperconverged Cluster Operator (HCO)?

    main

    The Hyperconverged Cluster Operator (HCO) is a unified operator designed to deploy and control KubeVirt along with several adjacent operators. It provides an opinionated deployment of the KubeVirt ecosystem by creating operator CustomResources (CRs) for its underlying components.

    Managed components include:

    • Containerized Data Importer (CDI)
    • Scheduling, Scale and Performance (SSP) Operator
    • Cluster Network Addons Operator
    • Node Maintenance Operator

  3. What is KubeVirt and how does it extend Kubernetes?

    main

    KubeVirt is a virtualization management add-on that extends Kubernetes by introducing new virtualization resource types, most notably the VirtualMachine type, via the Kubernetes Custom Resource Definitions (CRD) API.

    While the CRDs allow you to manage virtual machines using standard Kubernetes tools, the actual business logic and functionality (such as scheduling, launching, and migrating VMs) are provided by running KubeVirt controllers and agents on the cluster.

    With KubeVirt, you can declaratively manage the following VM lifecycle operations:

    • Create a predefined VM
    • Schedule a VM on a Kubernetes cluster
    • Launch a VM
    • Migrate a VM
    • Stop a VM
    • Delete a VM

  5. Use golden images in heterogeneous clusters

    main

    In clusters with different CPU architectures (e.g., amd64 and arm64), you can use a single golden image if it supports both architectures.

    Activation: This feature is in Alpha and is disabled by default. To enable it, set spec.featureGates.enableMultiArchBootImageImport: true in the HyperConverged CR.

    Custom/Modified Images: When adding or modifying a golden image where the source is different from the original common image, you MUST annotate the DataImportCronTemplate with ssp.kubevirt.io/dict.architectures containing a comma-separated list of supported architectures.

    Troubleshooting: If an image does not match any node architecture, the HyperConverged CR status will show a Deployed condition with reason UnsupportedArchitectures. You can check status.dataImportCronTemplates[].originalSupportedArchitectures to see what the image supports and compare it against status.nodeInfo.workloadsArchitectures.

    # Example: Annotating a custom image for multi-arch support
spec:
  dataImportCronTemplates:
  - metadata:
      name: kubevirt-hyperconverged
      annotations:
        ssp.kubevirt.io/dict.architectures: "amd64,arm64"
    spec:
      schedule: "0 */12 * * *"
      template:
        spec:
          source:
            registry:
                url: docker://my-private-registry/my-own-version-of-centos:8
      managedDataSource: centos-stream8

  6. Understand the Hyperconverged Cluster Operator (HCO)

    main

    The Hyperconverged Cluster Operator (HCO) is an implementation of the Operator pattern designed to manage multi-operator products. Its primary role is to orchestrate the deployment and management of several key components within a Kubernetes cluster:

    • KubeVirt: The core virtual machine management add-on.
    • Containerized Data Importer (CDI): Manages data import and disk management for VMs.
    • Virtual Machine import operator: Handles VM migration and importing.
    • Cluster Network Addons (CNA): Manages networking components for the hyperconverged stack.

  7. Monitor HCO system health status

    main

    You can monitor the overall health of the Hyperconverged Cluster Operator and its secondary resources using the kubevirt_hco_system_health_status metric or the recording rule cluster:kubevirt_hco_operator_health_status:count.

    Health values are represented as follows:

    • 0: Healthy
    • 1: Warning
    • 2: Error / Critical

    The recording rule cluster:kubevirt_hco_operator_health_status:count is more comprehensive as it aggregates both the kubevirt_hco_system_health_status metric and firing alerts that impact operator health.

  8. Enable or Disable HCO Feature Gates

    main

    Feature gates allow you to opt-in to features in alpha or beta phases, or opt-out of beta features. You can manage these by adding FeatureGate objects to the spec.featureGates set.

    Feature Gate Phases

    • alpha: Dev-preview. Disabled by default; must be enabled.
    • beta: Tech-preview. Enabled by default; can be disabled.
    • GA: Graduated. Always enabled; cannot be disabled.
    • deprecated: No longer supported; cannot be enabled.

    Notable Feature Gates

    • decentralizedLiveMigration (beta): Enables cross-cluster live migration.
    • declarativeHotplugVolumes (beta): Enables declarative volume hotplugging.
    • rebootPolicy (beta): Enables VM recreation upon guest OS reboot.
    • template (beta): Enables native in-cluster VM templating.
    • incrementalBackup (alpha): Enables changed block tracking and incremental backups.
    • alignCPUs (alpha): Requests additional dedicated CPUs for even parity.
    • containerPathVolumes (alpha): Allows VMs to access container paths via virtiofs.
    • deployObservabilityController (alpha): Deploys the virt-observability-controller for Prometheus metrics.

    Deprecated Feature Gates

    Do not use the following, as they have moved to dedicated configuration fields:

    • disableMDevConfiguration $\rightarrow$ use spec.virtualization.mediatedDevicesConfiguration.enabled
    • enableMultiArchBootImageImport $\rightarrow$ use spec.workloadSources.enableMultiArchBootImageImport
    • persistentReservation $\rightarrow$ use spec.storage.persistentReservationConfiguration.enabled

  10. Understand Functional Test coverage

    main

    Functional tests are located in the ../tests/func-tests directory. They are built as runnables and use the ginkgo framework. These tests validate high-level behaviors such as:

    • Priority class creation for critical workloads.
    • Workload and infra pod placement on specific nodes.
    • VMI (Virtual Machine Instance) creation, verification, and deletion with correct node placement.
    • ConsoleQuickStart object creation.

    To run functional tests using a Docker container, refer to the functest-container.md guide.

  12. Configure TLS Security Profiles for Hyperconverged Kubevirt

    main

    The Hyperconverged Cluster Operator (HCO) provides a mechanism to manage cluster-wide Crypto Policies. This allows administrators to define TLS security profiles that govern the TLS configuration for HCO-managed components.

    Platform Availability and Behavior

    • OCP/OKD Platforms: The TLS Security Profile configuration feature is natively available on OpenShift/OKD platforms.
    • Plain Kubernetes: On standard Kubernetes deployments, the HCO can be configured to optionally fine-tune or override configurations. If no custom values are expressed, the HCO can be designed to read the OpenShift API server's default configuration, or use a default intermediate profile for all managed components.

    Technical Implementation Details

    If you are extending or interacting with the TLS implementation, the configuration is expected to be performed via the standard Go crypto/tls library using the tls.Config struct. Key fields to tune according to the chosen tlsSecurityProfile include:

    • MinVersion
    • CipherSuites

    For example, an intermediate profile should align with recommended compatibility standards (such as those defined by Mozilla).