repository·main·Indexed 19 days ago
https://github.com/openshift/hypershiftMiddleware for hosting OpenShift control planes at scale, focusing on cost efficiency, rapid provisioning, and cross-cloud portability. Includes guides for provisioning Azure infrastructure via `hypershift create infra azure`, managing Azure workload identities and RBAC, and using the `cleanleaked` tool to detect and remove leaked AWS resources from CI test runs.
HyperShift is a middleware designed for hosting OpenShift control planes at scale. It addresses challenges related to cost, provisioning time, and cross-cloud portability by providing a strong separation of concerns between the management layer and the workload layer.
Hosted clusters are fully compliant OpenShift Container Platform (OCP) clusters and maintain compatibility with standard OCP and Kubernetes toolchains.
Self-managed Azure HyperShift allows you to deploy and manage OpenShift hosted control planes on an existing OpenShift management cluster running in Azure.
You can choose between two DNS approaches depending on your use case:
| Aspect | With External DNS | Without External DNS |
|---|---|---|
| Best For | Production, multi-cluster | Development, testing |
| API Server DNS | Custom (e.g., api-cluster.example.com) | Azure LoadBalancer (e.g., abc123.region.cloudapp.azure.com) |
| Setup Complexity | Higher (requires DNS zones, service principal) | Lower (minimal configuration) |
| Management | Fully automatic | Manual or Azure-provided |
Note: Self-managed Azure HostedClusters are available as a Developer Preview feature in OpenShift Container Platform 4.21.
HyperShift on Google Cloud Platform (GCP) uses a GKE Autopilot cluster as the management platform and Workload Identity Federation (WIF) for tokenless authentication.
Note: GCP HostedClusters are available as a TechPreview feature in OpenShift Container Platform 4.22.
HyperShift is composed of several specialized operators and services that manage the lifecycle of hosted clusters:
| Component | Role |
|---|---|
| HyperShift Operator | A singleton on the management cluster. It reconciles HostedCluster and NodePool resources, creates control plane namespaces, and deploys the CPO and CAPI controllers. |
| Control Plane Operator (CPO) | One instance per hosted cluster. It reconciles the HostedControlPlane and manages approximately 40 control plane components (like etcd and kube-apiserver) using the CPOv2 declarative framework. |
| Control Plane PKI Operator | Manages certificate rotation, CSR signing, and break-glass credential access for each hosted cluster. |
| Karpenter Operator | Manages Karpenter resources to handle node auto-scaling. |
| Ignition Server | An HTTPS server that generates and serves ignition configurations to worker nodes during the bootstrap process. |
Hypershift provides several methods for performing disaster recovery (DR) on HostedControlPlanes. Depending on your requirements, you can use:
HCPEtcdBackup feature gate.HyperShift supports multiple infrastructure platforms, including managed services and self-managed environments. Each platform implements a common Platform interface but uses different Cluster API (CAPI) providers, credential models, and networking primitives.
Supported Platforms:
The HyperShift E2E v2 testing framework follows a multi-phase lifecycle managed by a ci-operator. The process is platform-agnostic, though Azure is currently the reference implementation. The flow consists of four main phases:
ci-operator builds the hypershift-tests image and executes pre-steps (e.g., installing RBAC, setting up the nested management cluster, and installing the HyperShift operator).create-guests binary is executed. It creates multiple cluster variants (e.g., public, private, oauth-lb, upgrade, autoscaling, external-oidc) in parallel using the hypershift create cluster <platform> command. It waits for all clusters to reach the Available condition and for version rollouts to complete.run-tests binary is executed. It reads configuration from a SHARED_DIR and forks multiple test-e2e-v2 subprocesses. Each subprocess targets a specific cluster variant using the E2E_HOSTED_CLUSTER_NAME environment variable and a Ginkgo label filter.destroy-guests binary is executed to parallelly run hypershift destroy cluster <platform> for all created clusters, followed by destroying the nested management cluster.Key binaries available in the hypershift-tests image:
test-e2e-v2: The actual test runner.create-guests: Orchestrates cluster creation.run-tests: Orchestrates test execution.destroy-guests: Orchestrates cluster destruction.dump-guests: Collects artifacts from clusters.HyperShift is distributed across several repositories depending on your needs (core operator, community fork, or specific cloud/plugin integrations).
openshift/hypershift contains the HyperShift Operator, Control Plane Operator, CLI, API definitions, and tests.hypershift-community/hypershift is a community-maintained fork synced from the main repository.openshift/hypershift-oadp-plugin provides OpenShift API for Data Protection (OADP) support for HostedControlPlane backup and restore.openshift/aws-encryption-provider: AWS KMS-backed API server encryption.openshift/azure-kubernetes-kms: Azure Key Vault KMS plugin.openshift/apiserver-network-proxy: Konnectivity proxy for secure API server to node communication.openshift/aws-node-termination-handler: Handles EC2 instance shutdown gracefully.HyperShift allows the deployment of Hosted Clusters and Nodepools on the OpenStack platform.
Important Considerations:
dev-preview and is intended for development and testing purposes, not production use.HostedCluster is created on OpenStack, HyperShift automatically installs:Gophercloud SDK.HyperShift supports two deployment models on Microsoft Azure, depending on your management cluster platform and authentication requirements:
Refer to the specific guides for your chosen model to begin setup.
A NodePool is a scalable set of worker nodes associated with a HostedCluster.
Key characteristics:
NodePool remain consistent.NodePool is independent of the underlying machine architecture used by the control plane.replicas field.Konnectivity (also known as apiserver-network-proxy) is a component that facilitates secure network communication between the Kubernetes API server and cluster nodes.
In HyperShift's hosted control plane architecture, the control plane (e.g., running in a provider VPC) often lacks direct network access to worker nodes (e.g., running in a customer VPC). Konnectivity establishes a tunnel to bridge this gap, allowing the API server to perform tasks like:
exec).ValidatingWebhookConfiguration, MutatingWebhookConfiguration).Note: Konnectivity is not required for traffic initiated from worker nodes to the Kubernetes API server.