watchtower

Because watchtower uses boto3 to transmit logs, the act of sending logs can trigger DEBUG level messages from boto3, botocore, and urllib3. This can create a self-perpetuating loop of log messages.

watchtower.CloudWatchLogHandler automatically handles this by attaching a filter that:

1. Drops all DEBUG level messages from boto3, botocore, and urllib3.
2. Drops all messages from these libraries during flush() and close() operations.

Note that this filter only applies to the watchtower handler. If you have other handlers (like a console handler), they will still receive and print these DEBUG messages. For example, the following configuration will print botocore debug logs to stderr but will NOT send them to CloudWatch:

import watchtower, logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger()
logger.addHandler(watchtower.CloudWatchLogHandler())

When use_queues=True (the default), CloudWatchLogHandler operates using a producer-consumer model to minimize performance impact on your application:

1. Queuing: Each unique log stream gets its own queue.Queue.
2. Background Threading: A dedicated daemon thread is spawned for each stream to monitor the queue.
3. Batching Triggers: The background thread pulls messages from the queue and triggers a put_log_events call to AWS when one of these conditions is met:
   • send_interval has elapsed.
   • max_batch_size (in bytes) is reached.
   • max_batch_count (number of messages) is reached.
4. Flushing and Closing:
   • Calling .flush() forces the handler to send all currently queued messages.
   • Calling .close() signals the background threads to finish processing existing messages and then shut down. It is recommended to call .close() during application shutdown to ensure no logs are lost.

In a Django project, configure watchtower within the LOGGING dictionary in settings.py.

Important Note on Debugging: In the Django debug server (manage.py runserver), certain system loggers can cause deadlocks due to threading in the logging handler. It is recommended to set propagate: False for the django logger in development, or use production WSGI servers (like gunicorn or uwsgi) where this limitation does not apply.

import boto3

AWS_REGION_NAME = "us-west-2"

boto3_logs_client = boto3.client("logs", region_name=AWS_REGION_NAME)

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'root': {
        'level': 'DEBUG',
        'handlers': ['watchtower', 'console'],
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
        'watchtower': {
            'class': 'watchtower.CloudWatchLogHandler',
            'boto3_client': boto3_logs_client,
            'log_group_name': 'YOUR_DJANGO_PROJECT_NAME',
            'level': 'DEBUG'
        }
    },
    'loggers': {
        'django': {
            'level': 'DEBUG',
            'handlers': ['console'],
            'propagate': False
        }
    }
}

The CloudWatchLogFormatter (the default formatter for CloudWatchLogHandler) is designed to handle structured data. If you pass a dictionary to your logger instead of a string, the formatter will serialize it as a JSON object, which CloudWatch Logs can then parse and index.

Structured Logging Example

import logging
import watchtower

logger = logging.getLogger(__name__)
logger.addHandler(watchtower.CloudWatchLogHandler())

# Passing a dictionary creates a structured JSON log in CloudWatch
logger.critical({"request": "hello", "metadata": {"size": 9000}})

Including LogRecord Attributes

You can automatically include metadata from the Python LogRecord (like levelname, filename, process, etc.) inside the JSON message by setting add_log_record_attrs on the handler's formatter.

import logging
import watchtower

logger = logging.getLogger(__name__)
handler = watchtower.CloudWatchLogHandler()
# Enable forwarding of specific attributes
handler.formatter.add_log_record_attrs = ["levelname", "filename", "process", "thread"]
logger.addHandler(handler)

logger.critical({"request": "hello"})

This results in a CloudWatch event where the message field is a JSON string containing both your custom data and the requested metadata.