Envoy Gateway Documentation

The Envoy Gateway threat model analyzes security risks associated with deploying and operating Envoy Gateway within a multi-tenant Kubernetes (K8s) cluster.

In Scope:

• Envoy Gateway: Configuration, deployment, and operation (e.g., TLS certificate management, authentication, service-to-service routing).
• Kubernetes Cluster: Network policies, access control, and resource isolation for namespaces/tenants.
• Tenant Workloads: Interaction between tenant pods and Envoy Gateway.

Out of Scope:

• Underlying infrastructure security (e.g., EC2, S3).
• Non-Envoy related Kubernetes components.
• Implementation of Envoy Gateway as an egress traffic controller.

Threats in an Envoy Gateway environment are categorized into three distinct areas to help identify the source and impact of potential vulnerabilities:

• Container Security (CS): General threats to containerized applications, often originating from orchestrator misconfigurations or container vulnerabilities. These are not specific to Envoy Gateway.
• Gateway API (GW): Threats related to the Gateway API specification itself. These involve misconfigurations or excessive permissions on Gateway API resources (e.g., HTTPRoute or Gateway) that could affect any implementation.
• Envoy Gateway (EG): Threats specific to Envoy Gateway or Envoy Proxy configurations and features. Improperly configured features in these components could be leveraged to gain unauthorized access to protected resources.

Envoy Gateway manages resource watching for extensions to prevent race conditions and reconciliation loops.

1. Dynamic Watching: Envoy Gateway uses controller-runtime to create watches on Unstructured resources matching the group, version, and kind specified in the extensionManager.resources or extensionManager.policyResources configuration.
2. Data Transfer: When a watched resource changes, Envoy Gateway sends the resource as an Unstructured object to the extension via gRPC.
3. Validation: The extension is responsible for performing its own type validation on the received Unstructured data.
4. Policy Context: For policy resources targeting Gateway resources, the policy is provided as context to calls made to the HTTPListener hook.

Envoy Gateway handles the watching of custom resources introduced by extensions. Instead of the extension creating its own Kubernetes watches, Envoy Gateway uses controller-runtime to watch Unstructured resources matching the group, version, and kind configured in the extensionManager.resources field.

When a resource changes, Envoy Gateway sends the Unstructured resource to the extension via gRPC. This prevents race conditions and reconciliation loops. The extension is responsible for performing its own type validation on the received data.

The Envoy Gateway threat model analyzes security risks associated with deploying and operating Envoy Gateway within a multi-tenant Kubernetes (K8s) cluster.

In Scope:

• Envoy Gateway: Configuration, deployment, and operation (including TLS certificate management, authentication, and service-to-service routing).
• Kubernetes Cluster: Network policies, access control, and resource isolation for different namespaces/tenants.
• Tenant Workloads: Interaction between tenant pods and Envoy Gateway.

Out of Scope:

• Underlying infrastructure security (e.g., EC2, S3).
• Non-Envoy related Kubernetes components.
• Implementation of Envoy Gateway as an egress traffic controller.

Envoy Gateway uses Gateway API Extensions to provide features that are not part of the standard Kubernetes Gateway API. Unlike the legacy Ingress API which relied on non-type-safe annotations, Envoy Gateway extensions use type-safe Custom Resource Definitions (CRDs).

Envoy Gateway employs a policy attachment model. This allows you to apply custom policies to standard Gateway API resources (such as HTTPRoute or Gateway) without modifying the core API. This model ensures a separation of concerns and allows for easier configuration management across different teams.

These extensions are processed by the Envoy Gateway control plane and translated into xDS configurations for Envoy Proxy instances, removing the need to manage raw Envoy configuration directly.

You can accelerate TLS handshakes or protect private keys using specialized hardware by adding an Envoy Private Key Provider to Envoy.

Currently, two private key providers are available as Envoy contrib extensions:

• QAT: Available since Envoy 1.24.
• CryptoMB: Available since Envoy 1.20.

These providers leverage hardware capabilities (like Intel SPR/EMR Xeon server platforms) to speed up TLS operations.

To prevent resource accumulation, you can manage the lifecycle of cert-manager resources:

• Pruning CertificateRequest history: Set the cert-manager.io/revision-history-limit annotation on your Gateway to allow cert-manager to prune old history.
• Cleaning up Secrets: When removing a TLS listener from a Gateway, a Secret may linger. Use the cert-manager flag for cleaning up secrets when certificates are deleted to automate this.
• Manual Cleanup: To completely remove cert-manager and its resources:
  1. Uninstall cert-manager via Helm: helm uninstall --namespace cert-manager cert-manager.
  2. Delete the namespace: kubectl delete namespace/cert-manager.
  3. Manually remove the TLS listener from your Gateway and delete the associated Secret.

To prevent the leakage of OIDC client secrets and user password hashes due to misconfigured RBAC, follow these practices:

• Restrict Secret access: Ensure only authorized users and service accounts can access secrets. This is critical in namespaces where SecurityPolicy objects are configured, as these namespaces store sensitive credentials.
• Minimize RBAC permissions: Reduce the use of ClusterRoles and Roles that allow list and get operations on secrets.
• Audit permissions: Perform periodic audits of your RBAC permissions to ensure compliance with the principle of least privilege.