structlog: Structured Logging for Python

structlog is a production-ready structured logging solution for Python. It is designed around the concept of functions that take and return dictionaries, providing a flexible and high-performance way to handle log entries.

Key features include:

• Flexibility: You can choose to let structlog handle the output format or forward log entries to an existing system like the standard library's logging module.
• Output Formats: Out-of-the-box support for JSON, logfmt, and pretty console output (using ConsoleRenderer).
• Modern Compatibility: Supports asyncio, context variables, and type hints.
• Simplicity: Uses familiar APIs while maintaining a core logic based on dictionary manipulation.

A Bound Logger is an instance of structlog.typing.BindableLogger (typically structlog.BoundLogger or structlog.stdlib.BoundLogger) obtained via structlog.get_logger() or by using bind()/unbind() methods.

Key concepts:

• Context: A dictionary of key-value pairs attached to the logger. When logging, the context serves as the base for the Event Dictionary.
• Immutability: Bound loggers are immutable. You cannot modify the context of an existing logger instance. Instead, you must use .bind(**kwargs) or .unbind(*keys) to create a new logger instance with the updated context.
• API: The methods available on a bound logger constitute the user's logging API.

A bound logger is the primary interface in structlog obtained via structlog.get_logger(). It is called 'bound' because it maintains a context dictionary of key-value pairs that are merged into every log entry produced by that specific logger instance.

A bound logger consists of three components:

1. Context Dictionary: A collection of key-value pairs that are merged into each log entry. You can inspect this via structlog.get_context().
2. Processors: A list of functions called on every log entry. Each processor receives the output of the previous one.
3. Wrapped Logger: The component responsible for the final I/O (output). By default, this is structlog.PrintLogger, but it can be a standard library logging.Logger or any other object.

Important: Bound loggers do not perform I/O themselves; they manage context and proxy calls to the wrapped logger.

An Event Dictionary (or event dict) is the central data structure in structlog. It is a dictionary containing all information related to a single log entry.

Key characteristics:

• The event key is a special field that holds the name/message of the event.
• It is composed of the values bound to a Bound Logger's context merged with the key-value pairs passed directly to the logging method call.
• It is passed through a chain of Processors, which can add, modify, or remove key-value pairs.
• The final transformed dictionary is eventually passed to the Wrapped Logger for output.

A Processor is a callable used to transform log entries. Processors are organized into a chain where each processor receives the output of the previous one.

How they work:

1. A processor receives an event dictionary as an argument.
2. It performs transformations (adding, modifying, or removing keys).
3. It returns a new event dictionary.
4. The final processor in the chain produces the dictionary that is passed to the Wrapped Logger.

This pattern allows for highly composable and modular log transformations.

The Wrapped Logger is the component responsible for the actual output of the log entry. It is the final destination for the event dictionary after it has passed through all processors.

Common implementations:

• structlog.PrintLogger: Used for Native Logging (fast, does not depend on the standard library).
• logging.Logger: Used when integrating with the Python standard library logging module.

As of structlog 22.1.0, the structlog.threadlocal module is deprecated. You should use the structlog.contextvars module instead. contextvars is a more feature-rich superset that works correctly with thread-local data, asynchronous code, and greenlets.

Recommended replacements for structlog.threadlocal functions:

• structlog.contextvars.merge_contextvars
• structlog.contextvars.clear_contextvars
• structlog.contextvars.bind_contextvars
• structlog.contextvars.get_contextvars
• structlog.contextvars.get_merged_contextvars

A processor is a function that receives an event_dict and returns a modified event_dict. structlog uses processor chains to transform log entries in flight.

To use a custom processor, you must register it in the structlog.configure(processors=[...]) call.

import datetime
import structlog

# Define a custom processor
def timestamper(_, __, event_dict):
    event_dict["time"] = datetime.datetime.now().isoformat()
    return event_dict

# Configure structlog with the processor chain
structlog.configure(
    processors=[
        timestamper,
        structlog.processors.KeyValueRenderer()
    ]
)

structlog.get_logger().info("hi")
# Output: event='hi' time='2018-01-21T09:37:36.976816'

Every log entry passes through a processor pipeline. This is a chain of functions where each function receives a dictionary (event_dict), modifies it, and returns the new dictionary for the next processor in the chain.

Common tasks handled by built-in processors include:

• Collecting call stack information (structlog.processors.StackInfoRenderer)
• Formatting exception info (structlog.processors.format_exc_info)
• Adding timestamps (structlog.processors.TimeStamper)

def timestamper(logger, log_method, event_dict):
    """Add a timestamp to each log entry."""
    event_dict["timestamp"] = time.time()
    return event_dict

To maximize insight and minimize noise, aim for as few log entries per request as possible. This concept is known as Canonical Log Lines.

structlog facilitates this by allowing you to:

1. Bind data incrementally: Add context to loggers as a request progresses.
2. Use context-local loggers: Use loggers that are local to the current execution context (e.g., via contextvars) to ensure data is automatically attached to all logs within a specific scope.

In asyncio applications, you can prevent the processor chain from blocking the event loop by using structlog's non-standard asynchronous methods. These methods are prefixed with a (e.g., ainfo instead of info) and execute processing in a thread pool executor.

No extra configuration is required, and you can mix synchronous and asynchronous logging methods in the same application.

Note: This increases the computational cost per log entry but prevents logging from blocking your application. (Added in version 23.1.0)

import structlog

logger = structlog.get_logger()

# Asynchronous logging (non-blocking)
await logger.ainfo("event!")

# Regular synchronous logging
logger.info("event!")