aiologger

repository·main·Indexed 41 days ago

https://github.com/async-worker/aiologger

An asynchronous, non-blocking logging library for Python designed for asyncio applications. It provides a non-blocking replacement for the standard logging module, featuring AsyncStreamHandler for stdout/stderr, AsyncFileHandler (via aiofiles thread pool), and JsonLogger for single-line JSON output. Supports Python 3.7 through 3.10.

Tokens
4.1K
Snippets
17
Records
23
Agent score
100%

What's inside aiologger

  1. What is aiologger and when to use it

    main
    The standard Python logging module is I/O blocking, which can degrade the performance of asynchronous applications. aiologger provides a non-blocking, asynchronous logging implementation designed specifically for asyncio applications to ensure that logging operations do not interfere with the event loop's performance.
  2. Initialize Logger instances outside an event loop

    main

    Because actual stream initialization only occurs upon the first log call (lazy initialization), you can safely instantiate aiologger.Logger objects outside of a running asyncio event loop.

    from aiologger import Logger
    
    # This is safe to do before the loop starts
    logger = Logger.with_default_handlers(name='my-logger')
  3. Flatten log content to the root level

    main

    By default, all data passed to log methods is nested under the msg key. You can use the flatten parameter to move dictionary keys to the root of the JSON object.

    • As an instance attribute: Set flatten=True when initializing the logger to make all subsequent calls flattened.
    • As a method parameter: Pass flatten=True to a specific log call to flatten only that entry.

    Warning: Flattening can overwrite existing root-level keys (like logged_at).

    # As an instance attribute
    logger = JsonLogger.with_default_handlers(flatten=True)
    await logger.info({"status_code": 200})
    # Result: {"status_code": 200, "logged_at": "...", ...}
    
    # As a method parameter
    logger = JsonLogger.with_default_handlers()
    await logger.info({"status_code": 200}, flatten=True)
  4. Initialize aiologger.Logger outside an event loop

    main

    Because actual stream initialization only occurs upon the first log call (lazy initialization), you can safely initialize aiologger.Logger instances outside of a running event loop.

    import asyncio
    from aiologger import Logger
    
    # This is safe to do before the loop starts
    logger = Logger.with_default_handlers(name='my-logger')
    
    async def main():
        await logger.debug("debug")
        await logger.shutdown()
    
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
  5. Understand the async behavior of aiologger

    main

    While aiologger is designed for asynchronous applications, its non-blocking behavior depends on the destination of the logs:

    • stdout/stderr: Logging to standard output or standard error is fully async.
    • Files on disk: Logging to files is NOT truly async. Because there is no stable async I/O interface for disk files in the OS, aiologger uses aiofiles, which utilizes a Thread Pool to perform writes.

    If you require file logging, ensure you have aiofiles installed as an optional dependency.

  6. Use aiologger with synchronous syntax

    main

    If you cannot or do not want to use async/await throughout your codebase, you can use aiologger as if it were synchronous. Behind the scenes, logs are still generated asynchronously. To do this, replace your standard library logger instance with an instance of aiologger.Logger (e.g., using Logger.with_default_handlers()).

    Note: When using this mode, the output order of logs is NOT guaranteed because the calls are not being awaited.

    import asyncio
    from aiologger import Logger
    
    async def main():
        # Using synchronous-style calls inside an async function
        logger = Logger.with_default_handlers(name='my-logger')
    
        logger.debug("debug")
        logger.info("info")
        logger.warning("warning")
        logger.error("error")
        logger.critical("critical")
    
        await logger.shutdown()
    
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.run_forever()
  7. Use JsonLogger for single-line JSON output

    main

    The JsonLogger is a drop-in replacement for the default aiologger.Logger. It ensures that all log output is valid, single-line JSON. It can handle strings, dictionaries, and complex objects (like datetime or exceptions) by serializing them into the msg field of the JSON log entry.

    import asyncio
    from aiologger.loggers.json import JsonLogger
    
    async def main():
        logger = JsonLogger.with_default_handlers()
        await logger.info("Im a string")
        await logger.info({
            'key': 'value'
        })
        await logger.shutdown()
    
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()
  8. Migrate from standard lib logging to aiologger

    main

    To migrate from the standard Python logging module to aiologger:

    1. Change Imports: Replace from logging import getLogger with from aiologger import Logger.
    2. Change Instantiation: Replace logger = getLogger(__name__) with logger = Logger.with_default_handlers(name='your-name').
    3. Handle Lifecycle: Ensure you call await logger.shutdown() to properly close handlers when your application exits.
    4. Update Calls (Optional but Recommended): If log order is important, add await to your logging calls (e.g., await logger.info(...)).
  9. Migrate from standard library logging to aiologger

    main

    You can use aiologger as a drop-in replacement for the standard logging library if you want to keep your existing synchronous syntax. Behind the scenes, logs will still be generated asynchronously.

    To migrate, replace your standard logger initialization with aiologger.Logger.

    Standard Library approach:

    from logging import getLogger
    logger = getLogger(__name__)
    logger.info("message")

    aiologger (Synchronous-style) approach:

    from aiologger import Logger
    logger = Logger.with_default_handlers()
    logger.info("message")

    Note: When using the synchronous-style calls without await, the output order is not guaranteed.

    from aiologger import Logger
    
    logger = Logger.with_default_handlers()
    logger.info("info")