Argo Rollouts Gateway API Plugin

repository·main·Indexed 41 days ago

https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi

A plugin for Argo Rollouts that enables traffic management and progressive delivery (Canary, Blue-Green) via the Kubernetes Gateway API. It allows for dynamic traffic shifting by modifying HTTPRoute resources, with supported integrations including agentgateway and the AWS Gateway API Controller for Amazon VPC Lattice.

Tokens
40K
Snippets
100
Records
123
Agent score
51%

What's inside Argo Rollouts Gateway API Plugin

  1. What is the Ping-Pong feature and when to use it

    main

    The Ping-Pong feature enables zero-downtime rollouts for long-lived TCP and gRPC connections.

    Why use it?

    In a standard canary rollout, the canaryService and stableService use selector swaps at promotion time. For load balancers that track connections at the target-group level (like AWS NLB), this swap triggers target deregistration and re-registration, which drops all in-flight connections.

    How it works

    Instead of swapping selectors, Ping-Pong uses two persistent services (pingService and pongService) whose selectors never change. The rollout controller alternates which one is considered "stable" via the status.canary.stablePingPong field. The plugin then shifts traffic weight between these two services via the Gateway API route. Because no Kubernetes Service selectors are modified during promotion, connections are not dropped.

    Requirement: This feature requires Argo Rollouts v1.10 or later.

  2. How Header Based routing works

    main

    By default, Argo Rollouts performs canary traffic splitting at the request level using random percentages. This can cause inconsistent user experiences (e.g., a user seeing different UI elements on subsequent requests).

    Header Based routing allows you to direct specific users to the canary pods by using a specific HTTP header. When a request contains the designated header, the plugin instructs the Gateway API to route that request to the canary pods instead of using the random weight-based split. This provides deterministic routing for specific clients, such as internal testers or specific user segments.

  3. How the plugin handles multiple rules in a single HTTPRoute

    main

    Since version 0.15.0, the plugin supports HTTPRoute objects that contain multiple rules (e.g., different path prefixes).

    When useHeaderRoutes: true is set for an HTTPRoute, the plugin injects one managed header rule per source rule within that route. This ensures that the canary header applies to all paths defined in the HTTPRoute.

    Because the Gateway API requires unique rule names, the plugin uses a naming convention for the injected rules: original-name-indexN (e.g., canary-header for the first rule and canary-header-1 for the second).

  4. Compatibility requirements for Gateway API versions and Plugin versions

    main

    When using the Argo Rollouts Gateway plugin, the version of the plugin you use must match the API version supported by your Gateway API implementation:

    • Plugin version 0.2.0: Uses v1beta1 resources. This is required for 0.x implementations of the Gateway API.
    • Plugin version 0.3.0 and upwards: Requires a v1.0+ implementation and uses gateway.networking.k8s.io/v1 resources.

    If your provider is still using a 0.x version of the Gateway API, you must use plugin version 0.2.0.

  5. Automatically discover routes using label selectors

    main

    Instead of explicitly naming every route, you can use label selectors to automatically discover and manage routes. This is ideal for dynamic environments or when managing a large number of routes.

    How to use label selectors

    1. Label your routes: Add specific labels to your HTTPRoute (or other route types) resources.
    2. Configure the Rollout: Use the corresponding selector property (e.g., httpRouteSelector) in the argoproj-labs/gatewayAPI plugin configuration to match those labels.

    Supported Selector Types

    • httpRouteSelector: For HTTPRoute resources
    • grpcRouteSelector: For GRPCRoute resources
    • tcpRouteSelector: For TCPRoute resources

    You can use multiple selectors simultaneously in a single Rollout.

    apiVersion: argoproj.io/v1alpha1
    kind: Rollout
    metadata:
      name: rollouts-demo
      namespace: default
    spec:
      strategy:
        canary:
          trafficRouting:
            plugins:
              argoproj-labs/gatewayAPI:
                httpRouteSelector:
                  matchLabels:
                    app: my-app
                    canary-enabled: "true"
                namespace: default
          steps:
          - setWeight: 10
          # ...
  6. How the Argo Rollouts plugin system works

    main

    The Gateway API plugin functions as an RPC server that is executed within the same pod as the Argo Rollouts controller.

    Lifecycle and Configuration:

    1. The Argo Rollouts controller reads a ConfigMap named argo-rollouts-config from its namespace during startup.
    2. If present, the controller loads the plugins defined in that ConfigMap.
    3. Crucial: This loading process happens only once at startup. If you modify the ConfigMap or create it after the controller is running, you must restart the controller for changes to take effect.
    4. The controller uses the ConfigMap to locate plugins, downloads them, and executes them as separate RPC servers.
    5. When specific Rollout events occur (e.g., SetWeight), the controller makes remote procedure calls to the plugin's RPC server and waits for a response.
  7. Configure Contour GatewayClass and Gateway

    main

    Define a GatewayClass using the projectcontour.io/gateway-controller controller name and a Gateway object in the projectcontour namespace to trigger Contour to provision an Envoy instance.

    ---
    apiVersion: gateway.networking.k8s.io/v1
    kind: GatewayClass
    metadata:
      name: contour
    spec:
      controllerName: projectcontour.io/gateway-controller
    ---
    apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    metadata:
      name: contour
      namespace: projectcontour
    spec:
      gatewayClassName: contour
      listeners:
        - name: http
          protocol: HTTP
          port: 80
          allowedRoutes:
            namespaces:
              from: All
  8. Configure GatewayClass and Gateway for Kong

    main

    Set up the Kong infrastructure by creating a GatewayClass and a Gateway resource. The GatewayClass uses the konghq.com/kic-gateway-controller controller name.

    GatewayClass (gatewayclass.yml):

    apiVersion: gateway.networking.k8s.io/v1
    kind: GatewayClass
    metadata:
      name: kong
      annotations:
        konghq.com/gatewayclass-unmanaged: 'true'
    spec:
      controllerName: konghq.com/kic-gateway-controller

    Gateway (gateway.yml):

    apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    metadata:
      name: kong
    spec:
      gatewayClassName: kong
      listeners:
      - name: proxy
        port: 80
        protocol: HTTP
    apiVersion: gateway.networking.k8s.io/v1
    kind: GatewayClass
    metadata:
      name: kong
      annotations:
        konghq.com/gatewayclass-unmanaged: 'true'
    spec:
      controllerName: konghq.com/kic-gateway-controller
    ---
    apiVersion: gateway.networking.k8s.io/v1
    kind: Gateway
    metadata:
      name: kong
    spec:
      gatewayClassName: kong
      listeners:
      - name: proxy
        port: 80
        protocol: HTTP
  9. Configure Gateway and HTTPRoute for traffic splitting

    main

    To enable traffic splitting, you must create a Gateway resource using the kgateway GatewayClass and an HTTPRoute that references both your stable and canary services.

    1. Create the Gateway:

    apiVersion: gateway.networking.k8s.io/v1beta1
    kind: Gateway
    metadata:
      name: argo-rollouts-gateway
      namespace: argo-rollouts
    spec:
      gatewayClassName: kgateway
      listeners:
        - protocol: HTTP
          name: http
          port: 80

    2. Create the HTTPRoute: Attach the route to the Gateway and define backendRefs for both the stable and canary services.

    apiVersion: gateway.networking.k8s.io/v1beta1
    kind: HTTPRoute
    metadata:
      name: argo-rollouts-http-route
      namespace: argo-rollouts
    spec:
      parentRefs:
        - name: argo-rollouts-gateway
          namespace: argo-rollouts
      rules:
        - matches: []
          backendRefs:
            - name: argo-rollouts-stable-service
              port: 80
            - name: argo-rollouts-canary-service
              port: 80
    apiVersion: gateway.networking.k8s.io/v1beta1
    kind: Gateway
    metadata:
      name: argo-rollouts-gateway
      namespace: argo-rollouts
    spec:
      gatewayClassName: kgateway
      listeners:
        - protocol: HTTP
          name: http
          port: 80
    ---
    apiVersion: gateway.networking.k8s.io/v1beta1
    kind: HTTPRoute
    metadata:
      name: argo-rollouts-http-route
      namespace: argo-rollouts
    spec:
      parentRefs:
        - name: argo-rollouts-gateway
          namespace: argo-rollouts
      rules:
        - matches: []
          backendRefs:
            - name: argo-rollouts-stable-service
              port: 80
            - name: argo-rollouts-canary-service
              port: 80
  10. Install Traefik with Gateway API support

    main

    To use Traefik as a Gateway API implementation for Argo Rollouts, you must first install the Gateway API CRDs and then install Traefik via Helm with the kubernetesGateway provider enabled.

    1. Install Gateway API CRDs:
    kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml
    1. Install Traefik: Use the Traefik Helm chart (version 40.0.0) and enable the Kubernetes Gateway provider.

    2. Verify Installation: Wait for the deployment to be available and ensure the GatewayClass named traefik is created and shows ACCEPTED: True in the kubectl get gatewayclass output.

    # Install Gateway API CRDs
    kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.1/standard-install.yaml
    
    # Install Traefik
    helm repo add traefik https://traefik.github.io/charts
    helm repo update
    helm install traefik traefik/traefik \
      --set providers.kubernetesGateway.enabled=true \
      --version 40.0.0 \
      --namespace traefik \
      --create-namespace
    
    # Wait for availability
    kubectl wait --timeout=5m -n traefik deployment/traefik --for=condition=Available
    
    # Verify GatewayClass
    kubectl get gatewayclass
  11. Run the plugin locally for development

    main

    To develop and debug the plugin locally, follow these steps:

    1. Start your local Kubernetes cluster.
    2. Configure the plugin location: Create a ConfigMap named argo-rollouts-config in the Argo Rollouts controller's namespace (use default if running locally). In the manifest, use a file:// directive to point to your local build:
      file://<path to the local build>
    3. Prepare Argo Rollouts CRDs:
      kubectl create namespace argo-rollouts
      kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
      Note: Delete the in-cluster Argo Rollouts controller deployment after this step, as you will run the controller locally.
    4. Run the controller locally:
      cd ~/go/src/github.com/argoproj/argo-rollouts
      go run ./cmd/rollouts-controller/main.go

    Once running, the controller will find your local plugin and run it as an RPC server. You can attach any Go debugger to the process to set breakpoints.

    # 1. Create ConfigMap with file:// directive
    # 2. Install CRDs
    kubectl create namespace argo-rollouts
    kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
    
    # 3. Run controller locally
    cd ~/go/src/github.com/argoproj/argo-rollouts
    go run ./cmd/rollouts-controller/main.go