kuberesolver

repository·master·Indexed 20 days ago

https://github.com/sercand/kuberesolver

A gRPC name resolver that uses the Kubernetes API to discover service endpoints, enabling client-side load balancing and zero-downtime deployments. It watches endpoint changes via the Kubernetes API instead of relying on standard DNS, supporting the kubernetes:// URI scheme and providing Prometheus metrics for monitoring resolution health.

Tokens
3.7K
Snippets
15
Records
21
Agent score
71%

What's inside kuberesolver

  1. Register kuberesolver for gRPC

    master

    To use kuberesolver to resolve Kubernetes service addresses, you must register it with the gRPC resolver registry before calling grpc.NewClient.

    For standard in-cluster usage, use kuberesolver.RegisterInCluster(). This uses a lightweight internal Kubernetes client to find service endpoints without significantly increasing binary size.

    // Import the module
    import "github.com/sercand/kuberesolver/v6"
    
    // Register kuberesolver to grpc before calling grpc.NewClient
    kuberesolver.RegisterInCluster()
  2. Enable Client Side Load Balancing with kuberesolver

    master

    To achieve zero-downtime deployments and connect to all available service endpoints, you must pass a loadBalancingPolicy via grpc.WithDefaultServiceConfig. Without this, gRPC will not create subconnections for each individual endpoint provided by the resolver.

    Example using round_robin:

    import (
        "google.golang.org/grpc"
        "google.golang.org/grpc/credentials/insecure"
    )
    
    // ...
    
    cc, err := grpc.NewClient(
        "kubernetes:///service:grpc", 
        grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`),
        grpc.WithTransportCredentials(insecure.NewCredentials()),
    )
  3. Watch for endpoint updates using the watchInterface

    master

    The watchInterface provides a mechanism to observe and react to changes in endpoints. It is used to stream Event objects that represent additions, modifications, deletions, or errors.

    To use a watcher, you consume events from the channel returned by ResultChan(). When the channel is closed, it indicates that either an error occurred or Stop() was called, and the watcher has been cleaned up.

    Key Methods:

    • ResultChan() <-chan Event: Returns a read-only channel that receives Event objects. The channel is closed when the watch ends.
    • Stop(): Terminates the watch, closes the underlying stream, and closes the ResultChan().
    // Example of consuming events from a watchInterface
    for event := range watcher.ResultChan() {
        switch event.Type {
        case Added:
            // Handle new endpoint
        case Modified:
            // Handle update
        case Deleted:
            // Handle removal
        case Error:
            // Handle error event
        }
    }
  4. Understand the Event and EndpointSlice data structures

    master

    The Event struct is the primary unit of notification. It wraps an EndpointSlice, which contains the actual network information (endpoints and ports) for a specific resource.

    • Event: Contains the Type (EventType) and the Object (EndpointSlice).
    • EndpointSlice: A collection of Endpoints and Ports available for a service.
    • EndpointSliceList: A wrapper for a slice of EndpointSlice objects.
    type Event struct {
    	Type   EventType     `json:"type"`
    	Object EndpointSlice `json:"object"`
    }
    
    type EndpointSlice struct {
    	Endpoints []Endpoint
    	Ports     []EndpointPort
    }
    
    type EndpointSliceList struct {
    	Items []EndpointSlice
    }
  5. Supported Kubernetes resolver target formats

    master

    The kuberesolver supports several URI formats for resolving Kubernetes services. The resolver parses the service name, namespace, and port from these strings.

    FormatExampleDescription
    kubernetes:///service.namespace:portkubernetes:///my-svc.my-ns:8080Uses the endpoint path.
    kubernetes://namespace/service:portkubernetes://my-ns/my-svc:8080Uses host as namespace and path as service.
    kubernetes://service.namespace:portkubernetes://my-svc.my-ns:8080Uses host for service and namespace.

    Port Resolution Logic:

    • If the port is a number (e.g., :8080), it resolves to that specific port.
    • If the port is a string (e.g., :http), it attempts to match the port by its name in the Kubernetes EndpointSlice.
    • If no port is specified, it defaults to the first available port in the EndpointSlice.
  6. Register kuberesolver with a custom schema

    master

    If you want to use a different URI scheme (e.g., my-scheme://) instead of the default kubernetes://, use RegisterInClusterWithSchema. This allows you to define how the gRPC dialer identifies the resolver.

    Note that you must provide a valid schema string.

    import "github.com/sercand/sercand/kuberesolver"
    
    // Register with a custom scheme
    kuberesolver.RegisterInClusterWithSchema("my-custom-scheme")
  7. Register kuberesolver with gRPC

    master

    To use kuberesolver as a gRPC name resolver, you must register its builder with the gRPC resolver registry. You can register it using the default kubernetes scheme or a custom schema.

    If you are running inside a Kubernetes cluster, use RegisterInCluster() to automatically use the kubernetes scheme and an in-cluster Kubernetes client.

    import "github.com/sercand/sercand/kuberesolver"
    
    // Register with the default 'kubernetes://' scheme
    kuberesolver.RegisterInCluster()
  8. Configure RBAC permissions for kuberesolver

    master
    If your Kubernetes cluster uses RBAC, the service account running your application must have GET and WATCH permissions for the endpointslices resource to allow kuberesolver to discover service endpoints.
  9. Use kuberesolver with custom schema

    master

    If you need to use a schema name other than the default kubernetes, use RegisterInClusterWithSchema(schema) during your application startup.

    // Example of registering with a custom schema
    kuberesolver.RegisterInClusterWithSchema("my-custom-schema")
  10. Resolve Kubernetes services using kuberesolver URI schemes

    master

    Once registered, you can use the kubernetes:// scheme in your grpc.NewClient target string. The resolver supports several formats for specifying services and namespaces.

    Note: The cluster_name part of a fully qualified domain name (e.g., test.default.svc.cluster.local) is supported for compatibility but is not actually used to resolve service endpoints.

    // Standard formats
    kubernetes:///service-name:8080
    kubernetes:///service-name:portname
    kubernetes:///service-name.namespace:8080
    kubernetes:///service-name.namespace.svc.cluster_name
    kubernetes:///service-name.namespace.svc.cluster_name:8080
    
    // Alternative formats
    kubernetes://namespace/service-name:8080
    kubernetes://service-name:8080/
    kubernetes://service-name.namespace:8080/
    kubernetes://service-name.namespace.svc.cluster_name
    kubernetes://service-name.namespace.svc.cluster_name:8080
  11. Initialize an in-cluster K8sClient

    master

    Use NewInClusterK8sClient() to create a client configured for running inside a Kubernetes cluster. This constructor automatically loads the service account token and CA certificate from the standard Kubernetes secret paths (/var/run/secrets/kubernetes.io/serviceaccount/).

    It also sets up a background file watcher to automatically update the client's bearer token if the service account token file is rotated or updated by Kubernetes.

    Requirements:

    • The environment variables KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined (standard for in-cluster pods).
    • The service account token and CA cert must be present at their default locations.
    client, err := kuberesolver.NewInClusterK8sClient()
    if err != nil {
        log.Fatal(err)
    }