cloudflare-operator

The Cloudflare Operator is a Kubernetes Operator designed to automate the deployment and management of Cloudflare Tunnels and DNS records for Kubernetes Service resources (supporting HTTP, TCP, and early-access UDP).

Built using operator-sdk, it provides a dynamic way to run the cloudflared daemon on Kubernetes. Key capabilities include:

• Tunnel Management: Create and use existing Tunnels for Cloudflare for Teams via Custom Resources (CR/CRD). This includes managing Cloudflare API credentials via Secrets, running scaled cloudflared Deployments, and managing associated ConfigMaps. Tunnels can be scoped to either the Cluster or a Namespace.
• TunnelBinding Controller: Automates the connection between a Tunnel and a Service. It updates the cloudflared ConfigMap, restarts the cloudflared Deployment to apply changes, and creates a proxied CNAME DNS entry in Cloudflare. It also handles cleanup (reversing these actions) when a TunnelBinding is deleted using Kubernetes Finalizers.

The operator uses a controller-based architecture to bridge Kubernetes resources with Cloudflare services.

1. Tunnel Resource: Defines the core cloudflared infrastructure. It uses a Kubernetes Secret for Cloudflare API Tokens/Keys and manages a cloudflared Deployment and its corresponding ConfigMap. Tunnels can be defined at the Cluster or Namespace level.
2. TunnelBinding Resource: Acts as the glue between a Tunnel and a Kubernetes Service. When a TunnelBinding is created, the controller:
   • Updates the cloudflared ConfigMap to include the new Service.
   • Restarts the cloudflared Deployment.
   • Creates a proxied CNAME in Cloudflare pointing to the tunnel.
3. Lifecycle Management: The operator uses Kubernetes Finalizers to ensure that when a TunnelBinding is deleted, the DNS entries and configuration changes are cleaned up in Cloudflare and Kubernetes respectively.

You can use both direct service routing and reverse proxy routing within the same tunnel. This is useful for migrations or when specific endpoints need to bypass the proxy.

Important: The cloudflare-operator does not currently deterministically sort wildcards to the end of the cloudflared configuration file. Because cloudflared processes routes in order, a wildcard route (e.g., *.<domain>) placed before a specific route will prevent the specific route from being reached.

Workaround: Prefix the metadata.name of your wildcard TunnelBinding with zz- to ensure it is sorted to the end of the configuration file.

apiVersion: networking.cfargotunnel.com/v1alpha1
kind: TunnelBinding
metadata:
  # Prefix with zz- to ensure this wildcard route is processed LAST
  name: zz-ingress-nginx
subjects:
  - name: wildcard
    spec:
      fqdn: "*.<domain>"
      target: https://ingress-nginx-controller.ingress-nginx.svc.cluster.local:443
tunnelRef:
  kind: ClusterTunnel
  name: example-tunnel
---
apiVersion: networking.cfargotunnel.com/v1alpha1
kind: TunnelBinding
metadata:
  name: authelia
subjects:
  - name: authelia
tunnelRef:
  kind: ClusterTunnel
  name: example-tunnel

To deploy a configuration where cloudflared routes to multiple applications using TunnelBinding, follow these steps.

Prerequisites:

• kubectl installed.
• Cloudflare authentication Secret deployed.
• cloudflare-operator deployed.
• A Tunnel or ClusterTunnel already deployed.

Deployment Steps:

1. Deploy the backend applications: This creates the necessary Deployments and Services for your apps.

   kubectl apply -f manifests/whoami-1/
   kubectl apply -f manifests/whoami-2/

2. Deploy the TunnelBinding: This resource tells the operator how to route traffic from the tunnel to your services.

   kubectl apply -f manifests/cloudflare-operator/tunnel-binding.yaml

3. Verify connectivity: The operator uses the service name and the ClusterTunnel domain to automatically create DNS records. For example, if your services are whoami-1 and whoami-2, you should be able to access them at whoami-1.example.com and whoami-2.example.com.

To route internet traffic to your cluster, you must deploy a Tunnel or ClusterTunnel resource. This resource manages the deployment of cloudflared pods that establish an outbound connection to Cloudflare's edge servers.

Tunnel vs ClusterTunnel

• Tunnel: A namespaced resource. Any tunnelBinding referencing it must exist in the same namespace.
• ClusterTunnel: A cluster-scoped resource. It can be referenced by tunnelBinding resources from any namespace.

Prerequisites

1. kubectl installed.
2. Cloudflare authentication secret deployed to the operator.
3. cloudflare-operator deployed in your cluster.

Deployment Steps

1. Prepare Manifests: Choose either manifests/tunnel.yaml or manifests/cluster-tunnel.yaml. Replace the following placeholders with your actual values:

   • <email-address>: The email address associated with your Cloudflare zone.
   • <domain>: The domain associated with the Cloudflare zone.
   • <secret-name>: The name of the Kubernetes secret containing your Cloudflare credentials.
   • <account-id>: Your Cloudflare account ID.

2. Apply the Manifest:

   # For a namespaced Tunnel:
   kubectl apply -f manifests/tunnel.yaml
   
   # OR for a cluster-wide ClusterTunnel:
   kubectl apply -f manifests/cluster-tunnel.yaml

3. Verify Deployment: Ensure the resource is created and that the operator has generated the corresponding configmap and deployment in the cloudflare-operator-system namespace.

Once verified, you can proceed to set up routing using tunnelBinding resources.

The TunnelBinding resource is used to map Kubernetes services to a Cloudflare Tunnel. It replaces the older method of using service annotations. A TunnelBinding consists of subjects (the target services to be tunneled) and a tunnelRef (the specific tunnel to use).

Subject Configuration

Each entry in the subjects list defines a service to be exposed. The default kind is Service. Key fields include:

• name: The name of the service.
• fqdn: The Fully Qualified Domain Name to be used for the service (e.g., mysvc.example.com).
• protocol: The protocol used (e.g., http).
• target: The internal cluster URL/address of the service (e.g., http://svc01.ns.svc.cluster.local:8080).
• caPool: (Optional) A custom CA certificate.
• noTlsVerify: (Optional) Boolean to skip TLS verification.

Tunnel Reference (tunnelRef)

Specifies which tunnel the subjects should be bound to. Supported kind values are Tunnel or ClusterTunnel.

DNS Update Control

You can control whether the controller automatically updates DNS records using the disableDNSUpdates field in tunnelRef.

• If false (default): The controller manages DNS updates.
• If true: The controller will not update DNS. You must manually create CNAME entries pointing to your tunnel domain (format: tunnel-id.cfargotunnel.com). You can find your tunnel ID via kubectl get clustertunnel/tunnel <tunnel-name>.

apiVersion: networking.cfargotunnel.com/v1alpha1
kind: TunnelBinding
metadata:
  name: svc-binding
subjects:
  - kind: Service # Default
    name: svc01
    spec:
      fqdn: mysvc.example.com
      protocol: http
      target: http://svc01.ns.svc.cluster.local:8080
      caPool: custom.crt
      noTlsVerify: false
  - name: svc02  # Points to the second service
tunnelRef:
  kind: Tunnel # Or ClusterTunnel
  name: k3s-tunnel
  disableDNSUpdates: false