Argo Rollouts Gateway API Plugin
repository·main·Indexed 41 days ago
https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapiA plugin for Argo Rollouts that enables traffic management and progressive delivery (Canary, Blue-Green) via the Kubernetes Gateway API. It allows for dynamic traffic shifting by modifying HTTPRoute resources, with supported integrations including agentgateway and the AWS Gateway API Controller for Amazon VPC Lattice.
What's inside Argo Rollouts Gateway API Plugin
- The Argo Rollouts Gateway API plugin is an extension for Argo Rollouts that enables Progressive Delivery (such as Canary or Blue-Green deployments) using the Kubernetes Gateway API. It allows Argo Rollouts to manage traffic shifting by interacting with Gateway API resources instead of traditional service-based traffic management.
What is the Ping-Pong feature and when to use it
mainThe Ping-Pong feature enables zero-downtime rollouts for long-lived TCP and gRPC connections.
Why use it?
In a standard canary rollout, the
canaryServiceandstableServiceuse selector swaps at promotion time. For load balancers that track connections at the target-group level (like AWS NLB), this swap triggers target deregistration and re-registration, which drops all in-flight connections.How it works
Instead of swapping selectors, Ping-Pong uses two persistent services (
pingServiceandpongService) whose selectors never change. The rollout controller alternates which one is considered "stable" via thestatus.canary.stablePingPongfield. The plugin then shifts traffic weight between these two services via the Gateway API route. Because no Kubernetes Service selectors are modified during promotion, connections are not dropped.Requirement: This feature requires Argo Rollouts v1.10 or later.
How Header Based routing works
mainBy default, Argo Rollouts performs canary traffic splitting at the request level using random percentages. This can cause inconsistent user experiences (e.g., a user seeing different UI elements on subsequent requests).
Header Based routing allows you to direct specific users to the canary pods by using a specific HTTP header. When a request contains the designated header, the plugin instructs the Gateway API to route that request to the canary pods instead of using the random weight-based split. This provides deterministic routing for specific clients, such as internal testers or specific user segments.
How the plugin handles multiple rules in a single HTTPRoute
mainSince version 0.15.0, the plugin supports
HTTPRouteobjects that contain multiple rules (e.g., different path prefixes).When
useHeaderRoutes: trueis set for anHTTPRoute, the plugin injects one managed header rule per source rule within that route. This ensures that the canary header applies to all paths defined in theHTTPRoute.Because the Gateway API requires unique rule names, the plugin uses a naming convention for the injected rules:
original-name-indexN(e.g.,canary-headerfor the first rule andcanary-header-1for the second).Compatibility requirements for Gateway API versions and Plugin versions
mainWhen using the Argo Rollouts Gateway plugin, the version of the plugin you use must match the API version supported by your Gateway API implementation:
- Plugin version 0.2.0: Uses
v1beta1resources. This is required for 0.x implementations of the Gateway API. - Plugin version 0.3.0 and upwards: Requires a
v1.0+implementation and usesgateway.networking.k8s.io/v1resources.
If your provider is still using a 0.x version of the Gateway API, you must use plugin version 0.2.0.
- Plugin version 0.2.0: Uses
Automatically discover routes using label selectors
mainInstead of explicitly naming every route, you can use label selectors to automatically discover and manage routes. This is ideal for dynamic environments or when managing a large number of routes.
How to use label selectors
- Label your routes: Add specific labels to your
HTTPRoute(or other route types) resources. - Configure the Rollout: Use the corresponding selector property (e.g.,
httpRouteSelector) in theargoproj-labs/gatewayAPIplugin configuration to match those labels.
Supported Selector Types
httpRouteSelector: ForHTTPRouteresourcesgrpcRouteSelector: ForGRPCRouteresourcestcpRouteSelector: ForTCPRouteresources
You can use multiple selectors simultaneously in a single Rollout.
apiVersion: argoproj.io/v1alpha1 kind: Rollout metadata: name: rollouts-demo namespace: default spec: strategy: canary: trafficRouting: plugins: argoproj-labs/gatewayAPI: httpRouteSelector: matchLabels: app: my-app canary-enabled: "true" namespace: default steps: - setWeight: 10 # ...- Label your routes: Add specific labels to your
How the Argo Rollouts plugin system works
mainThe Gateway API plugin functions as an RPC server that is executed within the same pod as the Argo Rollouts controller.
Lifecycle and Configuration:
- The Argo Rollouts controller reads a ConfigMap named
argo-rollouts-configfrom its namespace during startup. - If present, the controller loads the plugins defined in that ConfigMap.
- Crucial: This loading process happens only once at startup. If you modify the ConfigMap or create it after the controller is running, you must restart the controller for changes to take effect.
- The controller uses the ConfigMap to locate plugins, downloads them, and executes them as separate RPC servers.
- When specific Rollout events occur (e.g.,
SetWeight), the controller makes remote procedure calls to the plugin's RPC server and waits for a response.
- The Argo Rollouts controller reads a ConfigMap named
Configure Contour GatewayClass and Gateway
mainDefine a
GatewayClassusing theprojectcontour.io/gateway-controllercontroller name and aGatewayobject in theprojectcontournamespace to trigger Contour to provision an Envoy instance.--- apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: contour spec: controllerName: projectcontour.io/gateway-controller --- apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: contour namespace: projectcontour spec: gatewayClassName: contour listeners: - name: http protocol: HTTP port: 80 allowedRoutes: namespaces: from: AllConfigure GatewayClass and Gateway for Kong
mainSet up the Kong infrastructure by creating a
GatewayClassand aGatewayresource. TheGatewayClassuses thekonghq.com/kic-gateway-controllercontroller name.GatewayClass (
gatewayclass.yml):apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: kong annotations: konghq.com/gatewayclass-unmanaged: 'true' spec: controllerName: konghq.com/kic-gateway-controllerGateway (
gateway.yml):apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: kong spec: gatewayClassName: kong listeners: - name: proxy port: 80 protocol: HTTPapiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: kong annotations: konghq.com/gatewayclass-unmanaged: 'true' spec: controllerName: konghq.com/kic-gateway-controller --- apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: kong spec: gatewayClassName: kong listeners: - name: proxy port: 80 protocol: HTTPConfigure Gateway and HTTPRoute for traffic splitting
mainTo enable traffic splitting, you must create a
Gatewayresource using thekgatewayGatewayClass and anHTTPRoutethat references both your stable and canary services.1. Create the Gateway:
apiVersion: gateway.networking.k8s.io/v1beta1 kind: Gateway metadata: name: argo-rollouts-gateway namespace: argo-rollouts spec: gatewayClassName: kgateway listeners: - protocol: HTTP name: http port: 802. Create the HTTPRoute: Attach the route to the Gateway and define
backendRefsfor both the stable and canary services.apiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute metadata: name: argo-rollouts-http-route namespace: argo-rollouts spec: parentRefs: - name: argo-rollouts-gateway namespace: argo-rollouts rules: - matches: [] backendRefs: - name: argo-rollouts-stable-service port: 80 - name: argo-rollouts-canary-service port: 80apiVersion: gateway.networking.k8s.io/v1beta1 kind: Gateway metadata: name: argo-rollouts-gateway namespace: argo-rollouts spec: gatewayClassName: kgateway listeners: - protocol: HTTP name: http port: 80 --- apiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute metadata: name: argo-rollouts-http-route namespace: argo-rollouts spec: parentRefs: - name: argo-rollouts-gateway namespace: argo-rollouts rules: - matches: [] backendRefs: - name: argo-rollouts-stable-service port: 80 - name: argo-rollouts-canary-service port: 80Install Traefik with Gateway API support
mainTo use Traefik as a Gateway API implementation for Argo Rollouts, you must first install the Gateway API CRDs and then install Traefik via Helm with the
kubernetesGatewayprovider enabled.- Install Gateway API CRDs:
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yamlInstall Traefik: Use the Traefik Helm chart (version 40.0.0) and enable the Kubernetes Gateway provider.
Verify Installation: Wait for the deployment to be available and ensure the
GatewayClassnamedtraefikis created and showsACCEPTED: Truein thekubectl get gatewayclassoutput.
# Install Gateway API CRDs kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml # Install Traefik helm repo add traefik https://traefik.github.io/charts helm repo update helm install traefik traefik/traefik \ --set providers.kubernetesGateway.enabled=true \ --version 40.0.0 \ --namespace traefik \ --create-namespace # Wait for availability kubectl wait --timeout=5m -n traefik deployment/traefik --for=condition=Available # Verify GatewayClass kubectl get gatewayclassRun the plugin locally for development
mainTo develop and debug the plugin locally, follow these steps:
- Start your local Kubernetes cluster.
- Configure the plugin location: Create a ConfigMap named
argo-rollouts-configin the Argo Rollouts controller's namespace (usedefaultif running locally). In the manifest, use afile://directive to point to your local build:file://<path to the local build> - Prepare Argo Rollouts CRDs:
Note: Delete the in-cluster Argo Rollouts controller deployment after this step, as you will run the controller locally.kubectl create namespace argo-rollouts kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml - Run the controller locally:
cd ~/go/src/github.com/argoproj/argo-rollouts go run ./cmd/rollouts-controller/main.go
Once running, the controller will find your local plugin and run it as an RPC server. You can attach any Go debugger to the process to set breakpoints.
# 1. Create ConfigMap with file:// directive # 2. Install CRDs kubectl create namespace argo-rollouts kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml # 3. Run controller locally cd ~/go/src/github.com/argoproj/argo-rollouts go run ./cmd/rollouts-controller/main.go