repository·main·Indexed 42 days ago
https://github.com/nhairs/python-json-loggerA JSON Log Formatter for the Python Logging Package (version 4.2.0) that enables the production of machine-readable JSON logs. It facilitates easier ingestion into log aggregation tools by providing a JsonFormatter that can be integrated into standard Python logging configurations, including YAML-based fileConfig.
Adhere to the following naming patterns:
lowercase_with_underscores.pylowercaseCapWords (e.g., BaseJsonFormatter)UPPERCASE_WITH_UNDERSCORES (e.g., RESERVED_ATTRS)lowercase_with_underscores.The project follows semantic versioning and prioritizes backwards compatibility.
Compatibility Rules:
major.minor version of python-json-logger is supported. Backporting bug fixes or features to older version branches requires prior discussion with maintainers.There are three primary ways to add contextual information like Request IDs to your logs:
extra argument: Pass the ID directly into every log call. This is explicit but requires manual effort at every call site.logging.Filter: Create a filter that attaches the ID to the LogRecord object. This is highly compatible with standard Python logging machinery and works automatically for all logs handled by the logger.process_log_record: Create a custom formatter that injects the ID into the data dictionary. This modifies the JSON output without altering the original LogRecord object.# Solution 1: Using extra
logger.info("message", extra={"request_id": request_id})
# Solution 2: Using a Filter
class RequestIdFilter(logging.Filter):
def filter(self, record):
record.request_id = get_request_id()
return True
logger.addFilter(RequestIdFilter())
# Solution 3: Using process_log_record
class MyFormatter(JsonFormatter):
def process_log_record(self, log_data):
log_data["request_id"] = get_request_id()
return log_dataNew encoders are only considered if they meet high popularity thresholds. To be considered, an encoder must be:
Process: You MUST open an issue for discussion before creating a pull request to add a new encoder. For implementation reference, see orjson.py and msgspec.py in the source code.
The project uses automated tools to enforce code style and catch errors. Ensure these tools are run before committing code. Configuration for these tools is located in the following files:
pyproject.toml).pylintrc).mypy.ini).logging package. This makes logs significantly easier to parse and ingest into log aggregation tools. To get started, you can install the package via PyPI and integrate it into your existing logging configuration.By default, JsonFormatter only includes fields explicitly defined in the format string. To include all standard library LogRecord attributes (like lineno, funcName, etc.) without manually listing them in the format string, set reserved_attrs to an empty list. This makes the inclusion of these fields implicit and avoids version-specific manual configuration.
all_fields_formatter = JsonFormatter(reserved_attrs=[])Before submitting a pull request, you must ensure your code adheres to the project's style and passes all quality checks. Run the following commands in order:
black on the src and tests directories.pylint (with colorized output) on src and mypy on src and tests.pytest.# Format
black src tests
# Lint
pylint --output-format=colorized src
mypy src tests
# Tests
pytestYou can install the library via pip or directly from GitHub releases.
Note: This fork uses version >=3.0.0. To use pre-fork versions, specify python-json-logger<3.
# Install via pip
pip install python-json-logger
# Install a specific wheel from GitHub releases
pip install 'python-json-logger@https://github.com/nhairs/python-json-logger/releases/download/v3.0.0/python_json_logger-3.0.0-py3-none-any.whl'To develop on python-json-logger, use uv to install the development dependencies. It is recommended to use a virtual environment.
To install all development tooling, run:
uv sync --group devTo produce JSON output, create an instance of JsonFormatter and attach it to a logging handler (e.g., StreamHandler).
import logging
from pythonjsonlogger.json import JsonFormatter
logger = logging.getLogger()
logHandler = logging.StreamHandler()
formatter = JsonFormatter()
logHandler.setFormatter(formatter)
logger.addHandler(logHandler)All functions and methods must have an explicit return statement. If a function does not logically return a value, end it with return None or simply return to make intent clear.
def process_data(data: dict) -> None:
"""Processes the given data."""
# ... processing logic ...
print(data)
return # or return None