nfdump

JA4 fingerprinting requires flows that contain payload data. To ensure fingerprinting works, you must use an exporter capable of exporting payload data.

If using nfpcapd, ensure you include the -o fat,payload option. Alternatively, you can use exporters like yaf that support payload export.

nfpcapd <your options> -o fat,payload

nfdump separates collection from processing by storing all collected data to disk before analysis. Data is organized in a time-based directory structure, typically rotating files every 5 minutes.

Directory Structure: Files are organized by flow source (e.g., router name) within a base directory: /flow_base_dir/router1 /flow_base_dir/router2

File Naming Convention: Files follow the pattern nfcapd.YYYYMMDDhhmm. For example, nfcapd.200907110845 contains data from July 11th, 2009, at 08:45 onward. With a 5-minute rotation interval, there are 288 files generated per day.

nfdump handles sampling rates automatically. By default, the sampling rate is 1 (unsampled) or the value specified via the -s flag. However, if the NetFlow stream contains sampling information (announced in v9/IPFIX option templates or via the v5 header hack), that value takes precedence.

Key behaviors:

• Data Scaling: The number of bytes and packets in each NetFlow record is automatically multiplied by the sampling rate to provide accurate totals.
• Flow Count: The total number of flows is not changed by the sampling multiplier, as this is considered inaccurate for small vs. large flows.

Use nfdump to analyze data from a single file or by concatenating multiple files. The filter syntax is inspired by tcpdump and allows for complex filtering and aggregation. Output can be formatted as ASCII text or CSV for post-processing.

Basic analysis of a specific file:

nfdump -r /flow_base_dir/router1/nfcapd.202501011200

Filtering by source IP and destination port:

nfdump -r flowfile 'src ip 192.0.2.1 and dst port 443'

To run the NetFlow collector (nfcapd) in a Docker container, build the nfcapd target and run it with UDP port 9995 exposed. It is recommended to create a Docker volume named flows and mount it to /data inside the container to avoid permission issues when running as a non-root user. You can append any desired nfcapd arguments to the end of the docker run command.

# Build the nfcapd target
docker build -t nfcapd --target nfcapd -f extra/docker/Dockerfile .

# Create a volume to prevent permission issues
docker volume create flows

# Run the collector
docker run -it --rm --name=nfcapd -p 9995:9995/udp -v flows:/data nfcapd

By default, nfdump only includes the free JA4: TLS Client Fingerprinting module. To build the full suite of JA4 modules, you must fulfill the licensing requirements specified by FoxIO LLC and configure the build with the --enable-ja4 flag.

Refer to the official JA4 documentation for licensing details: https://github.com/FoxIO-LLC/ja4#licensing

./configure --enable-ja4 ..

The nfdump tool reads binary flow files, applies filters, and performs aggregation or enrichment.

Basic Reading

Read a specific flow file: nfdump -r <path_to_file>

Filtering and Aggregation

Use a filter syntax (similar to tcpdump) to select specific flows and aggregate them by fields (e.g., srcip, dstip).

Compression

To compress files on the fly during collection, use the -z option with nfcapd.

# View a specific flow file
nfdump -r /flow_base_dir/router1/nfcapd.202501011200

# Filter for a specific source IP and destination port, then aggregate by source and destination IP
nfdump -r flowfile 'src ip 192.0.2.1 and dst port 443' -A srcip,dstip

nfdump-1.7.x is compatible with files created by nfdump-1.6.18 or newer. If you have legacy files from earlier versions, use nfdump to convert them to the new format. Note that only nfdump can read these legacy files; other programs require the new format.

# Convert old flow file to new format
./nfdump -r old-flowfile -y -w new-flowfile

nfdump uses the GNU autotools 2.71 build system. You can build it for general use or for a specific package, or optimize it for your local system.

General Build

Use this for creating portable packages or general distribution.

Local System Build (Optimized)

If you are running the tools on the same machine where they are built, enable CPU-specific optimizations (-march=native) and link-time optimization (-flto) for better performance.

Note: Optimized builds are not recommended for portable binaries.

# General use
./autogen.sh
./configure
make
sudo make install

# Local system optimization
./autogen.sh
./configure --enable-native --enable-lto
make
sudo make install

To use the nfdump analysis tool in a Docker container, build the nfdump target. By default, this drops you into an interactive shell. To access collected flow data, create a Docker volume named flows and mount it to /data inside the container. You can append desired nfdump arguments to the docker run command.

# Build the nfdump target
docker build -t nfdump --target nfdump -f extra/docker/Dockerfile .

# Create a volume to prevent permission issues
docker volume create flows

# Run the tool
docker run -it --rm --name=nfdump -v flows:/data nfdump

The nfexpire utility manages the expiration of NetFlow data files based on their lifetime or the total volume of all flow files. It is designed to work with collectors like nfcapd, sfcapd, or nfpcapd.

nfexpire operates in two modes:

1. Maintenance Mode: Used to set expiration parameters (size, lifetime, watermark) which are stored in a .nfstat file within the data directory.
2. Expire Mode: Actually runs the deletion process based on the stored parameters or provided overrides.

Expiration can be triggered manually, via a cron job, or automatically by the collector process itself.