Blackhole C++ Logging Library

repository·master·Indexed 41 days ago

https://github.com/3hren/blackhole

A high-performance, attribute-based C++ logging library focused on structured logging. It treats log entries as collections of key-value pairs, supporting flexible formatting via String and JSON formatters, and multiple output sinks including Console, File, TCP, UDP, and Syslog. Features include a logger_facade for convenient API usage, dynamic initialization via JSON configuration, and a fluent Builder API for static initialization.

Tokens
3.9K
Snippets
12
Records
19
Agent score
3%

What's inside Blackhole

  1. Overview of Blackhole logging

    master
    Blackhole is an attribute-based logger designed for maximum performance. Unlike traditional loggers that primarily handle text messages, Blackhole treats every log record as a collection of key-value pairs called Attributes. This allows for structured logging that can be easily processed by destinations like Elasticsearch.
  2. Use experimental features with caution

    master
    Blackhole provides certain symbols within the experimental namespace. These features are considered super-unstable, do not adhere to semantic versioning, and may be changed or dropped without notice. Use them only when you need to test cutting-edge functionality and can tolerate breaking changes.
  3. Understand Blackhole Attributes

    master

    Attributes are the core concept of Blackhole. Every logging record is escorted by key-value pairs. This structured approach allows you to split a single log line into multiple searchable indexes.

    For example, a standard HTTP access log can be decomposed into attributes such as host, user, timestamp, method, uri, protocol, status, and elapsed.

  4. Configure the special `{...}` placeholder for custom attributes

    master

    The {...} placeholder prints all non-reserved (user-declared) attributes in reverse order of provision as key-value pairs. You can extend this with pattern, separator, or format extensions.

    Extension Types:

    • Pattern (:p): Defines how each key-value pair is formatted. Uses sub-placeholders {name} and {value}. Example: {{name}={value}:p}.
    • Separator (:s): Defines the character(s) separating the key-value pairs. Example: { :s} for tabs.
    • Format: An overall format specifier applied to the entire result (e.g., >50s for right-alignment with width 50).
    // Example: tab separated key-value pairs like id=42\tmethod=GET
    {...:{{name}={value}:p}{\t:s}}
  5. Use the logger_facade for convenient logging

    master

    The raw logger_t interface is optimized for high performance but can be inconvenient for complex logging tasks. To enable easy message formatting and attribute provisioning, wrap your logger with blackhole::logger_facade.

    Supported patterns with the facade:

    1. Simple logging: Pass a string directly.
    2. Attributes via initializer list: Provide key-value pairs using {{"key", value}} syntax.
    3. Pattern formatting: Use Python-like placeholder syntax (e.g., "{} {}") with runtime arguments.
    4. Combined formatting and attributes: Provide format arguments followed by an attribute_list as the final argument.
    // 1. Simple
    log.log(severity::warning, "message");
    
    // 2. With attributes
    log.log(severity::info, "message", {{"key", value}});
    
    // 3. With formatting
    log.log(severity::debug, "{} {}", arg1, arg2);
    
    // 4. Combined (attributes must be last)
    log.log(severity::error, "{} {}", arg1, arg2, attribute_list{{"key", value}});
  6. Configure Blackhole using the Factory (Dynamic Initialization)

    master

    The Factory API is the recommended way to configure Blackhole. It implements dependency injection, allowing configuration to be loaded from external sources like JSON. This is referred to as dynamic initialization.

    // Build the logger using a registry and a JSON builder.
    auto log = blackhole::registry::configured()
        ->builder<blackhole::config::json_t>(std::ifstream(argv[1]))
            // Build the logger named "root".
            .build("root");
  7. Initialize Blackhole using a JSON configuration

    master

    You can initialize the entire logging pipeline (including severity mapping, handlers, formatters, and sinks) from a configuration file using the blackhole::registry.

    Steps:

    1. Use blackhole::registry::configured() to start the builder.
    2. Specify the builder type (e.g., blackhole::config::json_t) and provide a file stream.
    3. Call .build("name") to create the inner logger.
    4. Wrap the resulting logger with blackhole::logger_facade to enable advanced features.
    #include <blackhole/config/json.hpp>
    #include <blackhole/extensions/facade.hpp>
    #include <blackhole/registry.hpp>
    #include <blackhole/root.hpp>
    #include <fstream>
    
    // ... severity enum definition ...
    
    auto inner = blackhole::registry::configured()
        ->builder<blackhole::config::json_t>(std::ifstream("config.json"))
        .build("root");
    
    auto log = blackhole::logger_facade<blackhole::root_logger_t>(inner);
  8. Configure the JSON Formatter

    master

    The JSON formatter produces structured JSON trees. It is highly configurable via routing, renaming, and mutation to support tools like Logstash or rsyslog.

    Core Capabilities:

    • Routing: Uses JSON Pointer to move attributes into specific branches of the JSON tree. Use * to route all remaining attributes to a location.
    • Renaming: Maps attribute names to new keys (e.g., message to @message). Renaming occurs after routing.
    • Mutation: Transform fields like timestamp (via strftime) or severity (via a mapping array) during tree construction.
    • Uniqueness: By default, duplicate keys are allowed for speed. Enable unique to ensure only the last inserted value for a key is kept (requires heap allocation).
  9. Configure Blackhole requirements and compilation

    master

    Requirements

    • Compiler: C++11, C++14, or C++17 (C++17 enables additional functionalities).
    • Dependencies: Boost.Thread (required for Thread Local Storage).

    Compilation Notes

    • RTTI: The library can be compiled without Runtime Type Information by using the -fno-rtti flag.
  10. Configure the String Formatter with patterns

    master

    The String formatter converts log records into human-readable strings using a Python-like syntax with {} placeholders. You can specify alignment, width, and type (string or numeric) within the placeholders.

    Key Features:

    • Timestamp Formatting: Use {timestamp:{spec}s} for UTC or {timestamp:{spec}l} for local time using strftime specifications. Use {timestamp:d} for microseconds since Unix epoch.
    • Alignment and Width: Use syntax like {severity:>7} to right-align the severity field with a width of 7.
    • Escaping: To include literal braces, use {{ or }}.
    • Custom Attributes: Any user-provided attribute can be placed in the pattern by name (e.g., {scope}).
    [{severity:>7}] [{timestamp:{%Y-%m-%d %H:%M:%S.%f}s}] {scope}: {message}
  11. Configure Blackhole using the Experimental Builder

    master

    The Builder API allows for static initialization, where the logging configuration is defined at compile time. This involves using specialized builders for each component (handlers, formatters, sinks) in a fluent interface.

    // Configure string/console handler and build the logger.
    auto log = blackhole::experimental::partial_builder<blackhole::root_logger_t>()
       // Add the blocking handler.
       .handler<blackhole::handler::blocking_t>()
           // Configure string formatter with pattern syntax.
           .set<blackhole::formatter::string_t>("{severity}, [{timestamp}]: {message}")
               .mapping(&sevmap)
               .build()
           // Configure console sink to write into stdout.
           .add<blackhole::sink::console_t>()
               .build()
           // Build the handler.
           .build()
       // Build the logger.
       .build();
  12. Build a JSON Formatter using the C++ Builder API

    master

    Use the blackhole::formatter::json_t::builder_t streaming API to construct complex JSON formatters in C++.

    auto formatter = blackhole::formatter::json_t::builder_t()
        .route("/fields", {"message", "severity", "timestamp"})
        .route("/other")
        .rename("message", "#message")
        .rename("timestamp", "#timestamp")
        .newline()
        .unique()
        .build();