Go kit

repository·master·Indexed 12 days ago

https://github.com/go-kit/kit

A comprehensive programming toolkit for building microservices and distributed systems in Go. It provides abstractions for transport, serialization, and RPC, featuring packages for structured logging, service instrumentation via metrics (supporting Prometheus, StatsD, and expvar), and authentication middleware for Basic Auth and JWT.

Tokens
20.2K
Snippets
72
Records
97
Agent score
97%

What's inside Go kit

  1. What is Go kit?

    master
    Go kit is a programming toolkit designed for building microservices (or elegant monoliths) in Go. It provides a set of packages and best practices to solve common problems in distributed systems and application architecture, allowing developers to focus on business logic rather than infrastructure concerns.
  2. Use the metrics package for service instrumentation

    master

    The package metrics provides a set of uniform interfaces for service instrumentation, allowing you to decouple your business logic from specific metrics implementations. It supports three primary metric types:

    • Counters: For values that only increase (e.g., total requests).
    • Gauges: For values that can go up or down (e.g., current number of goroutines).
    • Histograms: For observing the distribution of values (e.g., request duration).

    Go kit provides adapters for popular metrics backends including expvar, StatsD, and Prometheus.

  3. Use the log package for structured logging

    master

    The log package provides a minimal interface for structured logging. Instead of unstructured strings, use a key/value-oriented format to provide semantic information. This makes logs easier to parse and analyze as data.

    Unstructured (Avoid): log.Printf("HTTP server listening on %s", addr)

    Structured (Preferred): logger.Log("transport", "HTTP", "addr", addr, "msg", "listening")

    logger.Log("transport", "HTTP", "addr", addr, "msg", "listening")
  4. How JSON RPC works with Go-Kit

    master

    In Go-Kit, a JSON RPC server implements the standard http.Handler interface. It routes incoming requests to specific logic based on the method property of the JSON RPC Request Object.

    Each JSON RPC method is implemented using an EndpointCodec. An EndpointCodec consists of three parts:

    1. Decoder: A function that extracts the params from the JSON RPC request and converts them into a Go type to be passed to the endpoint.
    2. Endpoint: The core business logic (a standard Go-Kit Endpoint).
    3. Encoder: A function that takes the endpoint's output and converts it into a raw JSON message, which the server then wraps in a JSON RPC Response Object's result field.
  5. Core design goals of Go kit

    master

    Go kit is designed around several key architectural principles:

    • Heterogeneous SOA Compatibility: It is built to operate in environments where it must interact with services written in languages other than Go.
    • RPC-First: Remote Procedure Call (RPC) is treated as the primary messaging pattern.
    • Pluggable Components: It supports pluggable serialization and transport layers, meaning you are not limited to JSON over HTTP.
    • Infrastructure Agnostic: It is designed to operate within existing infrastructures without mandating specific tools or technologies.
  6. Implement Thrift bindings for Go kit services

    master
    To connect a Go kit service to a Thrift transport, you must write a binding layer. This layer performs a straightforward conversion between your domain-specific Go kit service interface and the generated Thrift definitions. Once the binding is implemented, it can be attached to a Thrift listener to serve requests. Because Go kit supports multiple transports simultaneously, you can expose the same service via Thrift and other protocols (like HTTP or gRPC) at the same time.
  7. Bridge JWT transport headers and context

    master

    To make jwt.NewParser and jwt.NewSigner work, you must bridge the gap between the transport layer (HTTP/gRPC headers) and the Go context.Context.

    Go kit provides helper functions that implement the RequestFunc interface for use in ClientBefore or ServerBefore options:

    • jwt.HTTPToContext(): Moves JWT from HTTP headers to context.
    • jwt.ContextToHTTP(): Moves JWT from context to HTTP headers.
    • jwt.GRPCToContext(): Moves JWT from gRPC metadata to context.
    • jwt.ContextToGRPC(): Moves JWT from context to gRPC metadata.
  8. Go kit design non-goals

    master

    To maintain focus, Go kit explicitly excludes the following:

    • Non-RPC Messaging Patterns: It does not currently support patterns like MPI, pub/sub, or CQRS.
    • Re-implementation: It avoids re-implementing functionality that can be provided by adapting existing software.
    • Operational Concerns: It does not provide opinions or implementations for deployment, configuration, process supervision, or orchestration.
  9. Redirect stdlib logger to Go kit logger

    master

    If you want logs from the standard library log package to follow your Go kit structured format, use log.NewStdlibAdapter.

    import (
    	"os"
    	stdlog "log"
    	kitlog "github.com/go-kit/kit/log"
    )
    
    func main() {
    	logger := kitlog.NewJSONLogger(kitlog.NewSyncWriter(os.Stdout))
    	// Redirect stdlib output to the kit logger adapter
    	stdlog.SetOutput(kitlog.NewStdlibAdapter(logger))
    	stdlog.Print("I sure like pie")
    }
    logger := kitlog.NewJSONLogger(kitlog.NewSyncWriter(os.Stdout))
    stdlog.SetOutput(kitlog.NewStdlibAdapter(logger))
    stdlog.Print("I sure like pie")
  10. Use Basic Authentication middleware

    master

    The auth/basic package provides a middleware that validates credentials from the HTTP Authorization header against a provided username and password pair.

    To ensure the middleware can access the authentication header from an incoming HTTP request, you must use httptransport.ServerBefore(httptransport.PopulateRequestContext) in your httptransport.NewServer configuration. This populates the request context with necessary information for the middleware to inspect the headers.

    import httptransport "github.com/go-kit/kit/transport/http"
    
    // ...
    
    httptransport.NewServer(
        AuthMiddleware(cfg.auth.user, cfg.auth.password, "Example Realm")(makeUppercaseEndpoint()),
        decodeMappingsRequest,
        httptransport.EncodeJSONResponse,
        httptransport.ServerBefore(httptransport.PopulateRequestContext),
    )
  11. Use Zipkin for distributed tracing in Go kit

    master

    Go kit provides native bindings to zipkin-go for distributed tracing. This is the preferred method when using Zipkin in a polyglot microservices environment.

    Instrumentation is available for the following transport layers:

    • kit/transport/http
    • kit/transport/grpc