zlog Documentation

repository·master·Indexed 23 days ago

https://github.com/hardysimpson/zlog

A high-performance, thread-safe logging library written in pure C, designed as a faster alternative to syslog and log4c. It features a syslog-style model using categories, rules, and formats, supporting multi-output routing and granular filtering. The library provides both a standard API for category management and a simplified dzlog API for convenience.

Tokens
9.9K
Snippets
23
Records
36
Agent score
32%

What's inside zlog

  1. Using wildcards for category matching

    master

    zlog supports wildcards to allow for flexible log routing across multiple categories:

    • * (All Categories): Matches any category. Useful for redirecting all errors from all components to a single file.
      • Example: *.error "/var/log/error.log" matches any category with an error level.
    • _ (Super Category): Matches a prefix. If you have categories my_cat and my_dog, the super category my_ will match both.
      • Example: my_.INFO >stdout; matches my_cat.INFO and my_dog.INFO.
    • ! (Exclusion): Used to exclude specific levels or patterns (detailed in section 5.5.2).
  2. How zlog differs from the log4j model

    master

    Unlike the log4j model, which typically enforces a one-to-one relationship between a logger in source code and a logger in a configuration file, zlog follows a more flexible model similar to syslog.

    In zlog, one category in the source code can correspond to multiple rules in the configuration file. This allows for:

    • Multi-output: Sending the same log message to different destinations (e.g., stdout and a file) with different levels.
    • Granular Filtering: Defining different rules for different levels within the same category. For example, DEBUG logs can go to a specific file while WARN logs go to stderr.
  3. Understand the zlog Syslog model

    master

    zlog is built on a 'syslog model' consisting of three core concepts:

    • Category: Specifies different kinds of log entries (represented as zlog_category_t * in code). Categories allow you to distinguish different types of logs within the same application.
    • Rule: Defines how a category's logs are handled (e.g., matching a level and directing it to an output).
    • Format: Describes the detailed log pattern, such as including timestamps, source files, or line numbers.
  4. How category matching works in rules

    master

    Category names consist of letters, digits, and underscores (_). Matching follows these patterns:

    • *.*: Matches all categories (e.g., aa, aa_bb, xx).
    • aa_.*: Matches a super-category and all its sub-categories (e.g., aa, aa_bb, aa_bb_cc).
    • aa.*: Matches only the specific category aa (does not match aa_bb).
    • !.*: Matches any category that has no other rule matched.
    | summarize | category string from configure file | category matches | category not matches |
    | --- | --- | --- | --- |
    | * matches all | *.* | aa, aa_bb, aa_cc, xx, yy ... | NONE |
    | string end with underline matches super-category and sub-categories | aa_.* | aa, aa_bb, aa_cc, aa_bb_cc | xx, yy |
    | string not ending with underline accurately matches category | aa.* | aa | aa_bb, aa_cc, aa_bb_cc |
    | ! matches category that has no rule matched | !.* | xx | aa(as it matches rules above) |
  5. How conversion specifiers and patterns work in zlog

    master

    zlog uses a conversion pattern to define the structure of log entries. A pattern is a string containing text and conversion specifiers. Each specifier starts with a percent sign (%) and is followed by optional format modifiers (like field width or justification) and a conversion character that defines the data type (e.g., date, level, thread ID).

    The pattern parser automatically detects the end of a specifier when it encounters the next conversion character. There is no need for explicit separators between text and specifiers.

    Example Pattern: "%d(%m-%d %T) %-5V [%p:%F:%L] %m%n"

    When calling zlog_info(c, "hello, zlog");, this pattern produces: 02-14 17:17:42 INFO [4935:test_hello.c:39] hello, zlog

  6. How level matching works in rules

    master

    Levels in the configuration file are case-insensitive. There are six default levels: DEBUG, INFO, NOTICE, WARN, ERROR, and FATAL.

    You can use the following expressions to match levels:

    • *: Matches all source levels.
    • aa.debug: Matches all logs where [source level] >= debug.
    • aa.=debug: Matches only when [source level] == debug.
    • aa.!debug: Matches all logs where [source level] != debug.
    | example expression | meaning               |
    | ------------------ | ---------------------- |
    | *                  | all [source level]    |
    | aa.debug           | [source level]>=debug |
    | aa.=debug          | [source level]==debug  |
    | aa.!debug          | [source level]!=debug  |
  7. Configure log file rotation

    master

    Rotation controls how log files are managed by size and count to prevent disk exhaustion.

    Syntax for file rotation: "(file path)", (size) * (count) ~ "(archive name)"

    • Size: The threshold that triggers rotation (e.g., 10MB).
    • Count: The number of archive files to keep. 0 means keep all. If not specified, all old logs are kept.
    • Archive Name: Must include #r for Rolling or #s for Sequence.

    Rolling (#r) behavior: When out.log reaches the limit, it is renamed to out.log.1. The next rotation moves out.log.1 to out.log.2, and so on. The oldest file has the highest serial number.

    Sequence (#s) behavior: When aa.log reaches the limit, it is renamed to aa.log.0. The next rotation moves aa.log.0 to aa.log.1, and so on.

    Examples:

    • "aa.log", 10MB: Simple rolling rotation.
    • "aa.log", 10MB * 3 ~ "aa.log.#r": Rotates at 10MB, keeps 3 archives, uses rolling naming.
  8. Use Mapped Diagnostic Context (MDC) for contextual logging

    master

    MDC (Mapped Diagnostic Context) is a thread-local key-value map. It allows you to attach metadata to log entries, which can then be included in the log format or used to dynamically generate log file paths. This is useful for distinguishing between different scenarios (like different customers) in concurrent processing.

    Key features:

    • Thread Safety: The MDC map belongs to the thread; calling zlog_put_mdc() in one thread does not affect others.
    • Format Integration: Use %M(key) in your configuration [formats] to print the value associated with key.
    • Dynamic Paths: Use %M(key) in your configuration [rules] to create separate log files based on the MDC value (e.g., "mdc_%M(customer_name).log";).

    API:

    • zlog_put_mdc(const char *key, const char *value): Sets the value for the specified key in the current thread.
    // Example usage
    zlog_put_mdc("myname", "Zhang");
    zlog_info(zc, "2.hello, zlog");
  9. How zlog categories and rules match

    master

    zlog uses a matching mechanism between source code categories and configuration file rules. A rule consists of a category, a level, an output channel, and a format.

    When you retrieve a category in your C code using zlog_get_category("my_cat"), zlog looks for rules in the [rules] section of the configuration file that match the string my_cat.

    Example Workflow:

    1. Source Code: zlog_info(c, "hello, zlog"); where c is a category named my_cat.
    2. Config Rule: my_cat.DEBUG >stdout; simple
    3. Matching: zlog checks if the log level (INFO) is greater than or equal to the rule level (DEBUG). Since INFO >= DEBUG, the log is sent to stdout using the simple format.
    4. Format Definition: [formats] simple = "%m%n" determines the final output string.
    zlog_category_t *c;
    c = zlog_get_category("my_cat");
    zlog_info(c, "hello, zlog");
  10. Link zlog in your C/C++ application

    master
    To use zlog in your application, include zlog.h in your source files. When compiling, you must provide the path to the header files using -I and the path to the library using -L. You must also link against zlog (-lzlog) and the pthread library (-lpthread).
  11. Implement user-defined output functions

    master

    You can take control of how logs are handled by defining a custom output function. In this mode, zlog is responsible for generating the log path and message dynamically based on your configuration, but you are responsible for the actual output, rotation, and cleanup actions.

    To implement user-defined output:

    1. Define a placeholder name in the [rules] section of your configuration file.
    2. Register a C function to that name using zlog_set_record().
    3. The function must accept a zlog_msg_t * pointer.

    Note: Implementing complex rotation logic (like size-based or time-based rotation) in a custom output function can be difficult in multi-process or multi-threaded environments.

    # 1. Define in configuration file
    [formats]
    simple = "%m%n"
    
    [rules]
    my_cat.* $myoutput, " mypath %c %d";simple
    # 2. Set the function in C
    #include "zlog.h"
    
    int output(zlog_msg_t *msg)
    {
        printf("[mystd]:[%s][%s][%ld]\n", msg->path, msg->buf, (long)msg->len);
        return 0;
    }
    
    int main(int argc, char** argv)
    {
        int rc;
        zlog_category_t *zc;
    
        rc = zlog_init("test_record.conf");
        if (rc) {
            return -1;
        }
    
        // Register the custom output function
        zlog_set_record("myoutput", output);
    
        zc = zlog_get_category("my_cat");
        if (!zc) {
            zlog_fini();
            return -2;
        }
    
        zlog_info(zc, "hello, zlog");
        zlog_fini();
        return 0;
    }