django-structlog Documentation

repository·main·Indexed 19 days ago

https://github.com/jrobichaud/django-structlog

A structured logging integration for Django that uses structlog to inject cohesive metadata, such as request and user IDs, into log entries. It provides RequestMiddleware for the request/response lifecycle, integration with Celery tasks via DjangoStructLogInitStep, and support for tracking Django management commands through DjangoCommandReceiver. The library includes a prefix-based configuration system (DJANGO_STRUCTLOG_) and various signals to customize metadata for request starts, completions, and failures.

Tokens
17.7K
Snippets
47
Records
61
Agent score
65%

What's inside django-structlog

  1. Overview of django-structlog

    main

    django-structlog is a structured logging integration for Django projects using structlog. It enhances standard logging by producing cohesive metadata (such as request_id and user_id) on every log entry, making it significantly easier to track events or incidents across logs.

    Supported Integrations

    • Django REST framework: Supported by default. Note that when using TokenAuthentication (or other DRF authentications), user_id will only appear in request_finished and request_failed logs rather than every log entry.
    • django-ninja: Supported by default.
    • Celery: Requires additional configuration (see specific Celery documentation).
  2. Celery Task Bound Metadata

    main

    When running within a Celery task, certain metadata keys are automatically bound to every log entry produced by that task. This metadata is also propagated to all child Celery tasks. Additionally, all metadata bound to the caller's logger is also included in the task's logger.

    KeyValue
    task_idUUID of the current task
    parent_task_idUUID of the parent's task (if any)
    | Key | Value |
    | :--- | :--- |
    | `task_id` | UUID of the current task |
    | `parent_task_id` | UUID of the parent's task (if any) |
  3. Understand the log structure for Django commands

    main

    When command logging is enabled, django-structlog automatically injects context into your logs.

    For standard commands, logs will include:

    • command_name: The full path to the command.
    • command_id: A unique UUID for the command execution.

    For nested commands (commands called from within other commands), the library also provides:

    • parent_command_id: The command_id of the parent command, allowing you to trace execution hierarchies.
    # Example of standard command logs
    $ python manage.py example_command bar
    2023-09-13T21:10:50.084368Z [info     ] command_started                [django_structlog.commands] command_name=django_structlog_demo_project.users.example_command command_id=be723d34-59f5-468e-9258-24232aa4cedd
    2023-09-13T21:10:50.085325Z [info     ] my log                         [django_structlog_demo_project.users.management.commands.example_command] command_id=be723d34-59f5-468e-9258-24232aa4cedd foo=bar
    2023-09-13T21:10:50.085877Z [info     ] command_finished               [django_structlog.commands] command_id=be723d34-59f5-468e-9258-24232aa4cedd
    
    # Example of nested command logs showing parent_command_id
    $ python manage.py example_command bar
    2023-09-15T00:10:10.467250Z [info     ] my log                         [django_structlog_demo_project.users.management.commands.example_command] command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed foo=bar
    2023-09-15T00:10:10.468176Z [info     ] command_started                [django_structlog.commands] baz=2 command_id=57524ccb-a8eb-4d30-a989-4e83ffdca9c0 command_name=django_structlog_demo_project.users.example_nested_command parent_command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed
  4. Upgrade to 9.0+: Type hints and drf-standardized-errors

    main

    Version 9.0+ introduced the following changes:

    • Type hints: The library now uses Python type hints and is validated with mypy --strict.
    • drf-standardized-errors: Unhandled exceptions when using drf-standardized-errors will now be intercepted and logged properly. If you use structlog-sentry, exceptions will propagate as expected.
  5. Filter specific logs from being recorded

    main

    You can prevent specific log events from being recorded by implementing a custom logging.Filter. This is useful for excluding noisy events like request_started from certain handlers.

    1. Define a filter class that checks the event key in the log message dictionary.
    2. Register the filter in your Django LOGGING settings.
    # your_project/logging/filters.py
    
    import logging
    
    class ExcludeEventsFilter(logging.Filter):
        def __init__(self, excluded_event_type=None):
            super().__init__()
            self.excluded_event_type = excluded_event_type
    
        def filter(self, record):
            if not isinstance(record.msg, dict) or self.excluded_event_type is None:
                return True  # Include the log message if msg is not a dictionary or excluded_event_type is not provided
    
            if record.msg.get('event') in self.excluded_event_type:
                return False  # Exclude the log message
            return True  # Include the log message
    
    
    # in your settings.py
    
    LOGGING = {
        'version': 1,
        'disable_existing_loggers': False,
        'handlers': {
            'console': {
                'class': 'logging.StreamHandler',
                'filters': ['exclude_request_started']
            },
        },
        'filters': {
            'exclude_request_started': {
                '()': 'your_project.logging.filters.ExcludeEventsFilter',
                'excluded_event_type': ['request_started']  # Example excluding request_started event
            },
        },
        'loggers': {
            'django': {
                'handlers': ['console'],
                'level': 'DEBUG',
            },
        },
    }
  6. Enable command logging for Django management commands

    main

    To enable structured logging for Django management commands, you must install django-structlog with the [commands] extra, which includes django-extensions. You then need to enable the feature in your Django settings and decorate your command's handle method with @signalcommand from django_extensions.management.utils.

    # 1. Install with command support
    pip install django-structlog[commands]
    # 2. Enable in settings.py
    DJANGO_STRUCTLOG_COMMAND_LOGGING_ENABLED = True
    # 3. Decorate your command
    import structlog
    from django.core.management import BaseCommand
    from django_extensions.management.utils import signalcommand
    
    logger = structlog.getLogger(__name__)
    
    class Command(BaseCommand):
        def add_arguments(self, parser):
            parser.add_argument("foo", type=str)
    
        @signalcommand
        def handle(self, foo, *args, **options):
            logger.info("my log", foo=foo)
            return 0
  7. Upgrade to 4.0+: Minimum requirements and Celery extras

    main

    Version 4.0+ dropped support for Django versions below 3.2.

    Minimum requirements:

    • Django 3.2+
    • Python 3.7+
    • structlog 21.4.0+
    • (optionally) celery 5.1+

    Celery users: You can install django-structlog with the celery extra to ensure compatibility with your version of Celery.

    django-structlog[celery]==4.0.0