nlbwmon Traffic Accounting Tool

repository·master·Indexed 42 days ago

https://github.com/jow-/nlbwmon

A lightweight Linux traffic accounting tool that uses conntrack netlink sockets to monitor bandwidth usage by IP, MAC, and Layer 7 protocols. It includes a daemon for collecting and storing statistics in a database and the nlbw client utility for querying data in human-readable, JSON, or CSV formats. Supports integration as a LEDE/OpenWrt package.

Tokens
1.9K
Snippets
7
Records
10
Agent score
52%

What's inside nlbwmon

  1. Overview of nlbwmon

    master

    nlbwmon is a Linux-based traffic accounting tool that monitors bandwidth usage by network hosts using netlink sockets to pull information from the Linux kernel's conntrack entries. It is highly efficient because it collects statistics directly from conntrack entries and is notified by the kernel when entries are destroyed.

    Key features:

    • Tracks traffic by IP Version (IPv4/IPv6), IP Address, MAC address, and Layer 7 protocol (via port numbers).
    • Stores statistics in a database.
    • Supports multiple database generations (default is 10) to retain historical data.
    • Allows custom protocol definitions to classify traffic; unmatched traffic is classified as 'Other'.
  2. Install nlbwmon as a LEDE package feed

    master

    To build nlbwmon within a LEDE build environment, add the repository to your feeds.conf and update your feeds.

    cp feeds.conf.default feeds.conf
    echo "src-git nlbwmon https://github.com/jow-/nlbwmon.git" >> feeds.conf
    ./scripts/feeds update nlbwmon
    ./scripts/feeds install nlbwmon
  3. Use the nlbw client utility

    master

    The nlbw client is a command-line utility used to query the nlbwmon daemon for network traffic statistics. It can output data in human-readable, JSON, or CSV formats. The client communicates with the daemon via a Unix socket.

    Available Commands

    • show: Displays traffic statistics in a human-readable table format.
    • json: Outputs statistics in JSON format.
    • csv: Outputs statistics in CSV format.
    • list: Lists available database timestamps.
    • commit: Triggers a commit in the daemon.

    Common CLI Options

    • -c <command>: Specify the command to run (show, json, csv, list, commit).
    • -S <socket>: Path to the nlbwmon control socket.
    • -t <YYYY-MM-DD>: Filter statistics by a specific date.
    • -g <fields>: Group by specified fields (e.g., ip,mac). Use - prefix to invert grouping logic if applicable.
    • -o <fields>: Order by specified fields. Use - prefix to sort in descending order (e.g., -rx_bytes).
    • -n: Use plain numbers instead of human-readable units (e.g., 1024 instead of 1.0K).
    • -p <protocol_db>: Path to the protocol database file.
    • -s <separator>: Set the CSV separator character.
    • -q <quote>: Set the CSV quote character.
    • -e <escape>: Set the CSV escape character.
    # Example: Show traffic for a specific date, grouped by IP and MAC, ordered by RX bytes descending
    # nlbw -c show -t 2023-10-27 -g ip,mac -o -rx_bytes
  4. Format and group nlbw output

    master

    When using the nlbw client, you can manipulate the output structure using grouping and ordering flags:

    • -g col[,col]: Group output by specified columns (e.g., mac, fam). Prefix a column name with - to invert the order.
    • -o col[,col]: Order output by specified columns. Prefix a column name with - to invert the order.
    • -n: Use plain numbers instead of human-readable units (K, M, G, etc.).
    # Group by MAC and Family, order by connections
    lbw -c show -g mac,fam -o conn
    
    # Group by MAC, order by connections (descending) and transfer (descending)
    lbw -c show -g mac -o -conn,-tx
  5. Use nlbw commands: show, json, csv, list, commit

    master

    The nlbw client supports several commands via the -c flag to define the output format and action:

    • show: Outputs statistics in a human-readable format.
    • json: Outputs statistics in JSON format.
    • csv: Outputs statistics in CSV format.
    • list: Lists available databases. Use the -t flag with a date to select a specific database for reading.
    • commit: Writes data currently stored in memory to the database file (useful before a system reboot).
  6. Configure the nlbwmon daemon

    master

    The nlbwmon daemon manages the collection and storage of bandwidth statistics. You can configure its behavior using command-line flags. If using LEDE/OpenWrt, settings can also be managed via UCI or /etc/config/nlbwmon.

    # Example of common daemon configuration flags
    # -i: interval to save in-memory DB to file (seconds)
    # -r: interval to poll conntrack entries (seconds)
    # -s: network subnet to monitor
    # -o: storage directory for database files
    # -p: path to protocol description file
    # -G: number of database generations to retain (default 10)
    # -I: accounting period interval (e.g., '1' for 1st of month, '2017-01-17/14' for every 14 days)
    # -P: preallocate maximum database size in memory
    # -Z: gzip compress archive databases
  7. Query statistics with the nlbw client

    master

    The nlbw client utility is used to query the nlbwmon daemon for current or historical statistics. It communicates via a Unix domain socket (default: /var/run/nlbwmon.sock).

    # Basic usage to show human-readable stats
    lbw -c show
    
    # Query a specific historical database using a date
    lbw -t 2017-02-01 -c show
    
    # Output in JSON format
    lbw -c json
    
    # Output in CSV format with custom separator
    lbw -c csv -s ","
  8. Use client_main() to launch the nlbw client utility

    master

    The client_main function serves as the primary entry point for the nlbw client utility. It accepts standard command-line arguments (argc, argv) and returns an integer exit code. This function is intended to be called to initialize and run the client logic, typically when embedding the utility into another application or providing a custom entry point.

    int client_main(int argc, char **argv);
  9. Integrate the nlbw client via client_main()

    master

    The client_main function is the entry point for the nlbw utility. It handles argument parsing, initializes protocol databases, and dispatches the requested command. If you are embedding this logic or wrapping it, ensure you provide the necessary argc and argv parameters.

    Note: The function relies on an internal opt structure (likely defined in client.h or nlbwmon.h) to store socket paths and protocol database locations.

    int client_main(int argc, char **argv);
  10. Reference: nlbw client data fields

    master

    When using the -g (group by) or -o (order by) options, you can specify the following field names:

    Field NameDescription
    familyIP version (IPv4/IPv6)
    protoProtocol number
    portDestination port
    macSource MAC address
    ipSource IP address
    connsConnection count
    rx_bytesReceived bytes
    rx_pktsReceived packets
    tx_bytesTransmitted bytes
    tx_pktsTransmitted packets
    hostCombined Host (MAC/IP)
    layer7Layer 7 protocol name