slog-multi

repository·main·Indexed 20 days ago

https://github.com/samber/slog-multi

A Go library for advanced handler composition in structured logging (slog). It provides tools to build complex logging pipelines including fanout, routing, failover, load balancing, and middleware chains. Features include built-in predicates for routing, error recovery to prevent application crashes, and inline handlers/middleware for rapid custom logic development. Compatible with Go 1.21 and later.

Tokens
12.8K
Snippets
36
Records
60
Agent score
68%

What's inside slog-multi

  1. Overview of slog-multi features

    main

    slog-multi provides advanced composition patterns for Go's structured logging (slog). It allows you to build sophisticated logging workflows by combining multiple handlers using various strategies:

    • Fanout: Distribute logs to multiple handlers in parallel.
    • Router: Conditionally route logs based on custom criteria.
    • First Match: Route logs to the first matching handler only.
    • Failover: High-availability logging with automatic fallback.
    • Load Balancing: Distribute load across multiple handlers.
    • Pipeline: Transform and filter logs with middleware chains.
    • Error Recovery: Graceful handling of logging failures.

    It also supports Inline Handlers and Inline Middleware for rapid development of custom logic.

  2. Implement custom middleware

    main

    A middleware in slog-multi is a function that takes an slog.Handler and returns a new slog.Handler.

    Important Requirement: When implementing custom middleware, the WithAttrs and WithGroup methods must return a new instance of your middleware rather than returning this (the current instance) to ensure proper immutability and handler chaining.

    type Middleware func(slog.Handler) slog.Handler
  3. Install slog-multi

    main

    Install the slog-multi package using go get. This library is compatible with Go version 1.21 and later.

    WARNING

    Use this library carefully. Complex logging workflows with multiple destinations and processing steps can introduce significant performance overhead in critical paths.

    go get github.com/samber/slog-multi
  4. Best practices for slog-multi

    main

    Performance

    • Fanout: Use slogmulti.Fanout() sparingly as broadcasting to many handlers can impact performance.
    • Sampling: Implement sampling strategies for high-volume logs.
    • Buffering: Use buffering for network-based handlers to mitigate latency.

    Reliability and Error Handling

    • Error Recovery: Always wrap handlers with RecoverHandlerError to prevent logging failures from crashing the application.
    • Failover: Use failover patterns for critical logging paths.

    Security

    • Redaction: Use middleware to remove PII (Personally Identifiable Information) and secrets (e.g., passwords, tokens).
    • Encryption: Use TLS for network-based handlers.
  5. What is a Middleware in slog-multi?

    main

    In slog-multi, Middleware is a functional abstraction defined as type Middleware func(slog.Handler) slog.Handler.

    It acts as a wrapper around a slog.Handler. When you pass a middleware to a composition function like Pipe, the middleware is executed to wrap the provided handler, allowing you to inject logic (like filtering or attribute modification) into the logging pipeline before the record reaches its final destination (the 'sink').

  6. Load balance logs using PoolHandler

    main

    The PoolHandler implements a load balancing strategy for slog.Handler instances. It distributes log records across multiple handlers using a round-robin approach with randomization. This is useful for increasing logging throughput, providing redundancy, or load balancing across multiple logging destinations.

    Key behaviors:

    • Distribution: Uses round-robin with randomization to prevent hot-spotting.
    • Enabled Check: Enabled returns true if at least one underlying handler is enabled for the specified level.
    • Error Handling: Handle attempts to distribute the record to a selected handler. If the selected handler is not enabled or fails, it continues through the pool. It returns nil if any handler successfully processes the record, or the last error encountered if all attempts fail.
    • Propagation: WithAttrs and WithGroup propagate attributes and groups to all child handlers in the pool.
    // Example: Distributing logs across three different handlers
    handler := slogmulti.Pool()(
        handler1, // Receives ~33% of records
        handler2, // Receives ~33% of records
        handler3, // Receives ~33% of records
    )
    logger := slog.New(handler)
  7. How RoutableHandler works

    main

    A RoutableHandler is a wrapper around a standard slog.Handler that adds conditional logic. It implements the slog.Handler interface, including Enabled, Handle, WithAttrs, and WithGroup.

    When Handle is called, the RoutableHandler checks its internal predicates. If the predicates match the record, it forwards the record to the underlying handler. It also correctly manages attribute accumulation and group hierarchies, ensuring that WithAttrs and WithGroup calls behave consistently with standard slog expectations.

  8. How FanoutHandler handles log levels and errors

    main

    The FanoutHandler implements the standard slog.Handler interface with specific logic for multi-destination logging:

    Log Level Checking (Enabled)

    A log level is considered enabled if at least one of the underlying handlers is enabled for that level. This ensures that if any destination is interested in a log, the fanout mechanism will attempt to deliver it.

    Error Handling (Handle)

    When distributing a record, FanoutHandler iterates through all handlers. If a handler is enabled, it calls its Handle method. If any handler returns an error, FanoutHandler collects all such errors and returns them as a single combined error using errors.Join.

  9. FirstMatch routing logic and behavior

    main

    The FirstMatchHandler follows a specific execution flow for every log record:

    1. Iteration: It loops through the registered handlers in the order they were provided.
    2. Predicate Matching: It calls isMatch(ctx, r) on each RoutableHandler.
    3. Level Validation: If a match is found, it checks Enabled(ctx, record.Level).
    4. Execution:
      • If the handler is matched and enabled, it executes the handler's Handle method and returns the result.
      • If the handler is matched but not enabled, it returns nil immediately (it does not fall through to the next handler).
    5. Fallback: If the loop completes without any matches, it returns nil.
  10. Recover from handler errors with `slogmulti.RecoverHandlerError()`

    main

    Use slogmulti.RecoverHandlerError() to prevent your application from crashing due to logging failures. It catches both panics and errors returned by handlers. You provide a callback function that is executed when a handler fails.

    recovery := slogmulti.RecoverHandlerError(
        func(ctx context.Context, record slog.Record, err error) {
            log.Println(err.Error())
        },
    )
    
    logger := slog.New(
        slogmulti.Pipe(recovery).Handler(sink),
    )
  11. Create inline handlers

    main

    Inline handlers allow you to implement the slog.Handler interface without defining a full struct. You can use NewHandleInlineHandler to implement only the Handle method, or NewInlineHandler to implement both Enabled and Handle.

    // Implement only Handle()
    mdw := slogmulti.NewHandleInlineHandler(
        func(ctx context.Context, groups []string, attrs []slog.Attr, record slog.Record) error {
            // Custom logic here
            return nil
        },
    )
    
    // Implement Enabled() and Handle()
    mdw := slogmulti.NewInlineHandler(
        func(ctx context.Context, groups []string, attrs []slog.Attr, level slog.Level) bool {
            return true
        },
        func(ctx context.Context, groups []string, attrs []slog.Attr, record slog.Record) error {
            return nil
        },
    )
  12. Route logs based on criteria with `slogmulti.Router()`

    main

    Use slogmulti.Router() to distribute logs to all matching slog.Handler instances based on custom predicates. Predicates can be based on log levels, attributes, or custom business logic.

    Common use cases include:

    • Environment-specific logging (dev vs prod)
    • Level-based routing (errors to Slack, info to console)
    • Business logic routing (user actions vs system events)
    logger := slog.New(
        slogmulti.Router().
            Add(slackChannelUS, recordMatchRegion("us")).
            Add(slackChannelEU, recordMatchRegion("eu")).
            Add(consoleHandler, slogmulti.LevelIs(slog.LevelInfo, slog.LevelDebug)).
            Handler(),
    )