httplog v3

repository·master·Indexed 43 days ago

https://github.com/go-chi/httplog

A high-performance, structured HTTP request logging middleware for Go web applications built on the standard log/slog package. It provides automatic log level assignment, panic recovery, and support for industry-standard schemas including ECS, OTEL, and GCP. Features include request/response body logging, custom attribute injection via SetAttrs, and a utility to generate shell-safe curl commands for replicating requests.

Tokens
4.5K
Snippets
14
Records
18
Agent score
53%

What's inside httplog

  1. Configure log schemas and modes

    master

    The library supports structured logging compatible with ECS, OTEL, and GCP. You can use schema objects to define how attributes are replaced in your slog.Handler.

    Example of creating a concise schema for local development:

    // logFormat is a schema that can be used in slog.Handler's ReplaceAttr
    logFormat := httplog.SchemaECS.Concise(isLocalhost)
    
    logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
        ReplaceAttr: logFormat.ReplaceAttr,
    }))
  2. Use predefined logging schemas (ECS, OTEL, GCP)

    master

    The httplog package provides pre-configured Schema instances to ensure your HTTP logs comply with industry standards. You can use these schemas with slog.Handler to automatically map standard log attributes to the specific field names required by different platforms.

    Available schemas:

    • SchemaECS: Elastic Common Schema (v9.0.0) for Elasticsearch/Elastic Stack.
    • SchemaOTEL: OpenTelemetry semantic conventions (v1.34.0).
    • SchemaGCP: Google Cloud Platform structured logging format.

    To use a schema, pass its ReplaceAttr method to an slog.Handler (such as slog.NewJSONHandler).

    // Example: Using the GCP schema with a JSON handler
    handler := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
        ReplaceAttr: SchemaGCP.ReplaceAttr,
    })
    logger := slog.New(handler)
  3. Handle client-side request abortions

    master
    If a client disconnects before the response is sent, the middleware detects the context cancellation. It logs the error using the ErrClientAborted value and sets the error type to ClientAborted in the logs.
  4. Use httplog.RequestLogger middleware

    master

    Integrate httplog into your router (Chi, Gin, Echo, or standard http.ServeMux) using httplog.RequestLogger. It requires a *slog.Logger and an *httplog.Options configuration object.

    Key configuration options in httplog.Options:

    • Level: Controls verbosity.
      • slog.LevelDebug: Log all responses (including OPTIONS).
      • slog.LevelInfo: Log responses (excluding OPTIONS).
      • slog.LevelWarn: Log 4xx and 5xx responses only (except for 429).
      • slog.LevelError: Log 5xx responses only.
    • Schema: A schema object (e.g., httplog.SchemaECS) used to format the log output via ReplaceAttr.
    • RecoverPanics: If true, recovers from panics in handlers/middlewares, returns HTTP 500, and logs the panic as an error.
    • Skip: A function func(req *http.Request, respStatus int) bool to filter out specific logs.
    • LogRequestHeaders: A slice of strings specifying which request headers to log.
    • LogResponseHeaders: A slice of strings specifying which response headers to log.
    • LogRequestBody: A predicate function to conditionally log the request body.
    • LogResponseBody: A predicate function to conditionally log the response body.
    r.Use(httplog.RequestLogger(logger, &httplog.Options{
        Level:           slog.LevelInfo,
        Schema:          logFormat,
        RecoverPanics:   true,
        Skip:            func(req *http.Request, respStatus int) bool { return respStatus == 404 },
        LogRequestHeaders: []string{"Origin"},
        LogRequestBody:  isDebugHeaderSet,
        LogResponseBody: isDebugHeaderSet,
    }))
  5. Add custom attributes to logs using SetAttrs

    master

    You can inject custom structured log attributes into the request context from within your own middlewares or handlers. These attributes will be included in the final HTTP log entry produced by httplog.

    Use httplog.SetAttrs(ctx, ...) to attach slog.Attr values to the context.

    r.Use(func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            ctx := r.Context()
    
            // Add custom attributes to the context
            httplog.SetAttrs(ctx, slog.String("user", "user1"))
    
            next.ServeHTTP(w, r.WithContext(ctx))
        })
    })
  6. Configure RequestLogger with Options

    master

    The Options struct allows fine-grained control over the logging behavior.

    Common configuration fields:

    • LogRequestBody: A function func(*http.Request) bool to determine if the request body should be logged.
    • LogResponseBody: A function func(*http.Request) bool to determine if the response body should be logged.
    • LogBodyContentTypes: A slice of strings (e.g., []string{"application/json"}) specifying which Content-Type headers allow body logging.
    • LogBodyMaxLen: Maximum number of bytes to log from the body before trimming.
    • Skip: A function func(r *http.Request, statusCode int) bool to prevent logging specific requests.
    • RecoverPanics: If true, the middleware recovers from panics, logs the error/stack trace, and returns an HTTP 500.
    • Schema: Defines the attribute keys used in the log (e.g., SchemaECS).
    • LogExtraAttrs: A function func(r *http.Request, body string, statusCode int) []slog.Attr to inject custom attributes into the log.
    opts := &httplog.Options{
    	LogRequestBody: func(r *http.Request) bool {
    		return r.Method == "POST"
    	},
    	LogBodyContentTypes: []string{"application/json"},
    	LogBodyMaxLen: 1024,
    	RecoverPanics: true,
    	Skip: func(r *http.Request, status int) bool {
    		return r.URL.Path == "/health"
    	},
    }
    
    middleware := httplog.RequestLogger(logger, opts)
  7. Add extra attributes with LogExtraAttrs

    master

    The LogExtraAttrs field in httplog.Options allows you to inject custom metadata into your logs based on the request and response state. This is useful for adding debugging information like curl commands for failed requests.

    Warning: Ensure you do not leak sensitive information (like Authorization headers) when implementing this function.

    // Log all requests with invalid payload as curl command.
    func(req *http.Request, reqBody string, respStatus int) []slog.Attr {
        if respStatus == 400 || respStatus == 422 {
            // Remove sensitive header before generating curl command
            req.Header.Del("Authorization")
            return []slog.Attr{slog.String("curl", httplog.CURL(req, reqBody))}
        }
        return nil
    }
  8. Set request errors using SetError

    master

    Use SetError to attach an error to the request log context. If the provided error is non-nil, it is added to the log attributes using the key error (defined by the ErrorKey constant). This allows you to record failures within your handlers so they are automatically captured in the final request log entry.

    err := doSomething()
    if err != nil {
    	// This attaches the error to the log context
    	httplog.SetError(r.Context(), err)
    	return err
    }
  9. Use RequestLogger middleware

    master

    The RequestLogger function returns a middleware that logs HTTP request details using a *slog.Logger. It captures request/response metadata, status codes, durations, and optionally request/response bodies.

    To use it, pass an existing *slog.Logger and an optional *Options struct to RequestLogger. If o is nil, default options are used.

    Key features include:

    • Automatic log level selection based on HTTP status code (e.g., 5xx is Error, 4xx is Warn).
    • Panic recovery and stack trace logging if RecoverPanics is enabled.
    • Support for different log schemas (e.g., SchemaECS).
    • Ability to log request and response bodies based on content type and size limits.
    import (
    	"log/slog"
    	"net/http"
    	"github.com/go-chi/chi/v5"
    	"github.com/go-chi/go-chi/httplog"
    )
    
    func main() {
    	logger := slog.Default()
    	// Use default options
    	middleware := httplog.RequestLogger(logger, nil)
    
    	r := chi.NewRouter()
    	r.Use(middleware)
    
    	r.Get("/hello", func(w http.ResponseWriter, r *http.Request) {
    		w.Write([]byte("world"))
    	})
    
    	http.ListenAndServe(":8080", r)
    }
  10. Set log attributes using SetAttrs

    master

    Use SetAttrs to attach structured slog.Attr values to the current request's log context. These attributes will be included in the log entry generated by the httplog middleware for that specific request. This is useful for adding request-specific metadata (like user IDs or transaction IDs) that you want to appear in your logs.

    // Inside a handler or middleware
    httplog.SetAttrs(r.Context(), slog.String("user_id", "123"), slog.Int("attempt", 1))
  11. Simplify logs for local development with Schema.Concise

    master

    The Concise method returns a simplified version of a schema. When concise is set to true, it strips away most request and response metadata, leaving only essential error and body information. This is useful for reducing log verbosity during local development.

    Fields retained in concise mode:

    • ErrorMessage
    • ErrorStackTrace
    • RequestHeaders
    • RequestBody
    • RequestBytesUnread
    • ResponseHeaders
    • ResponseBody
    // Create a simplified GCP schema for local development
    localSchema := SchemaGCP.Concise(true)