Go kit
repository·master·Indexed 12 days ago
https://github.com/go-kit/kitA comprehensive programming toolkit for building microservices and distributed systems in Go. It provides abstractions for transport, serialization, and RPC, featuring packages for structured logging, service instrumentation via metrics (supporting Prometheus, StatsD, and expvar), and authentication middleware for Basic Auth and JWT.
What's inside Go kit
- Go kit is a programming toolkit designed for building microservices (or elegant monoliths) in Go. It provides a set of packages and best practices to solve common problems in distributed systems and application architecture, allowing developers to focus on business logic rather than infrastructure concerns.
Use the metrics package for service instrumentation
masterThe
package metricsprovides a set of uniform interfaces for service instrumentation, allowing you to decouple your business logic from specific metrics implementations. It supports three primary metric types:- Counters: For values that only increase (e.g., total requests).
- Gauges: For values that can go up or down (e.g., current number of goroutines).
- Histograms: For observing the distribution of values (e.g., request duration).
Go kit provides adapters for popular metrics backends including
expvar,StatsD, andPrometheus.Use the log package for structured logging
masterThe
logpackage provides a minimal interface for structured logging. Instead of unstructured strings, use a key/value-oriented format to provide semantic information. This makes logs easier to parse and analyze as data.Unstructured (Avoid):
log.Printf("HTTP server listening on %s", addr)Structured (Preferred):
logger.Log("transport", "HTTP", "addr", addr, "msg", "listening")logger.Log("transport", "HTTP", "addr", addr, "msg", "listening")How JSON RPC works with Go-Kit
masterIn Go-Kit, a JSON RPC server implements the standard
http.Handlerinterface. It routes incoming requests to specific logic based on themethodproperty of the JSON RPC Request Object.Each JSON RPC method is implemented using an
EndpointCodec. AnEndpointCodecconsists of three parts:- Decoder: A function that extracts the
paramsfrom the JSON RPC request and converts them into a Go type to be passed to the endpoint. - Endpoint: The core business logic (a standard Go-Kit
Endpoint). - Encoder: A function that takes the endpoint's output and converts it into a raw JSON message, which the server then wraps in a JSON RPC Response Object's
resultfield.
- Decoder: A function that extracts the
Core design goals of Go kit
masterGo kit is designed around several key architectural principles:
- Heterogeneous SOA Compatibility: It is built to operate in environments where it must interact with services written in languages other than Go.
- RPC-First: Remote Procedure Call (RPC) is treated as the primary messaging pattern.
- Pluggable Components: It supports pluggable serialization and transport layers, meaning you are not limited to JSON over HTTP.
- Infrastructure Agnostic: It is designed to operate within existing infrastructures without mandating specific tools or technologies.
Implement Thrift bindings for Go kit services
masterTo connect a Go kit service to a Thrift transport, you must write a binding layer. This layer performs a straightforward conversion between your domain-specific Go kit service interface and the generated Thrift definitions. Once the binding is implemented, it can be attached to a Thrift listener to serve requests. Because Go kit supports multiple transports simultaneously, you can expose the same service via Thrift and other protocols (like HTTP or gRPC) at the same time.Bridge JWT transport headers and context
masterTo make
jwt.NewParserandjwt.NewSignerwork, you must bridge the gap between the transport layer (HTTP/gRPC headers) and the Gocontext.Context.Go kit provides helper functions that implement the
RequestFuncinterface for use inClientBeforeorServerBeforeoptions:jwt.HTTPToContext(): Moves JWT from HTTP headers to context.jwt.ContextToHTTP(): Moves JWT from context to HTTP headers.jwt.GRPCToContext(): Moves JWT from gRPC metadata to context.jwt.ContextToGRPC(): Moves JWT from context to gRPC metadata.
Go kit design non-goals
masterTo maintain focus, Go kit explicitly excludes the following:
- Non-RPC Messaging Patterns: It does not currently support patterns like MPI, pub/sub, or CQRS.
- Re-implementation: It avoids re-implementing functionality that can be provided by adapting existing software.
- Operational Concerns: It does not provide opinions or implementations for deployment, configuration, process supervision, or orchestration.
Redirect stdlib logger to Go kit logger
masterIf you want logs from the standard library
logpackage to follow your Go kit structured format, uselog.NewStdlibAdapter.import ( "os" stdlog "log" kitlog "github.com/go-kit/kit/log" ) func main() { logger := kitlog.NewJSONLogger(kitlog.NewSyncWriter(os.Stdout)) // Redirect stdlib output to the kit logger adapter stdlog.SetOutput(kitlog.NewStdlibAdapter(logger)) stdlog.Print("I sure like pie") }logger := kitlog.NewJSONLogger(kitlog.NewSyncWriter(os.Stdout)) stdlog.SetOutput(kitlog.NewStdlibAdapter(logger)) stdlog.Print("I sure like pie")Use Basic Authentication middleware
masterThe
auth/basicpackage provides a middleware that validates credentials from the HTTPAuthorizationheader against a provided username and password pair.To ensure the middleware can access the authentication header from an incoming HTTP request, you must use
httptransport.ServerBefore(httptransport.PopulateRequestContext)in yourhttptransport.NewServerconfiguration. This populates the request context with necessary information for the middleware to inspect the headers.import httptransport "github.com/go-kit/kit/transport/http" // ... httptransport.NewServer( AuthMiddleware(cfg.auth.user, cfg.auth.password, "Example Realm")(makeUppercaseEndpoint()), decodeMappingsRequest, httptransport.EncodeJSONResponse, httptransport.ServerBefore(httptransport.PopulateRequestContext), )Compile protobuf definitions to Go
masterOnce
protocis installed, use it to compile your.protoservice definition into Go code. Ensure your proto definition matches your service's go-kit (interface) definition.protoc add.proto --go_out=plugins=grpc:.Use Zipkin for distributed tracing in Go kit
masterGo kit provides native bindings to
zipkin-gofor distributed tracing. This is the preferred method when using Zipkin in a polyglot microservices environment.Instrumentation is available for the following transport layers:
kit/transport/httpkit/transport/grpc