ankerl::nanobench

Nanobench can provide detailed CPU statistics such as instructions (ins/op), cycles (cyc/op), instructions per cycle (IPC), branches (bra/op), and branch misses (miss%).

Note: These statistics are only available on Linux via perf events. On some systems, you may need to adjust perf_event_paranoid settings or use ACLs to allow unprivileged access to performance counters.

When running a benchmark, ankerl::nanobench prints a formatted table containing several performance metrics.

Key metrics include:

• ns/op: Nanoseconds per operation.
• op/s: Operations per second.
• err%: The measurement error percentage.
• ins/op: Average instructions executed per operation.
• cyc/op: Average CPU cycles per operation.
• IPC: Instructions Per Cycle.
• bra/op: Average branches per operation.
• miss%: Branch prediction miss rate.
• total: Total runtime of the benchmark.
• benchmark: The name of the benchmarked code block.

When benchmarking very fast code (like simple arithmetic), the compiler might optimize the code away if the result is not used, leading to incorrect results (e.g., :boom: iterations overflow.).

To prevent this, use ankerl::nanobench::doNotOptimizeAway() to ensure the compiler treats the result as if it were being used.

#include <nanobench.h>

int main() {
    ankerl::nanobench::Bench b("++x");
    uint64_t x = 0;
    b.run([&] {
        x += 1;
        ankerl::nanobench::doNotOptimizeAway(x);
    });
}

The library outputs a table containing detailed performance metrics for each benchmark. Key columns include:

• ns/op: Nanoseconds per operation.
• op/s: Operations per second.
• err%: Measurement fluctuation/error percentage.
• ins/op: Average instructions executed per operation.
• cyc/op: Average CPU cycles per operation.
• IPC: Instructions per cycle.
• bra/op: Number of branches per operation.
• miss%: Branch prediction miss percentage.
• total: Total runtime of the benchmark.
• benchmark: The name assigned to the benchmark.

Nanobench can automatically determine the Big O complexity of an algorithm by running the benchmark multiple times with different input sizes ($N$).

1. Use bench.complexityN(N) to record a benchmark run with a specific complexity parameter $N$.
2. After multiple runs with different $N$ values, use bench.complexityBigO() to get a sorted table of the best-fitting complexity functions (e.g., $O(1)$, $O(\log n)$, $O(n)$, etc.).

The table is sorted from the best approximation (lowest error) to the worst.

To use ankerl::nanobench, define ANKERL_NANOBENCH_IMPLEMENT before including <nanobench.h>. You can then create a ankerl::nanobench::Bench instance and use the .run() method to benchmark a lambda or function.

To prevent the compiler from optimizing away the code you are trying to measure, use ankerl::nanobench::doNotOptimizeAway(variable).

#define ANKERL_NANOBENCH_IMPLEMENT
#include <nanobench.h>

int main() {
    double d = 1.0;
    ankerl::nanobench::Bench().run("some double ops", [&] {
        d += 1.0 / d;
        if (d > 5.0) {
            d -= 5.0;
        }
        ankerl::nanobench::doNotOptimizeAway(d);
    });
}

Once you have generated pyperf compatible JSON files (e.g., result.json), you can use the Python pyperf module to perform various analyses.

Show Benchmark Statistics

To view statistical summaries of your benchmark runs, use the stats command:

python3 -m pyperf stats result.json

Show a Histogram

To visually identify outliers in your benchmark data, generate a histogram:

python3 -m pyperf hist result.json

Compare Results

To compare two different benchmark outputs (for example, comparing a standard implementation against a nanobench implementation), use the compare_to command:

python3 -m pyperf compare_to result_a.json result_b.json

You can integrate nanobench into your CMake project using FetchContent or as a git submodule.

Example CMakeLists.txt structure:

include(FetchContent)
FetchContent_Declare(
  nanobench
  GIT_REPOSITORY https://github.com/martinus/nanobench.git
  GIT_TAG v4.3.11
)
FetchContent_MakeAvailable(nanobench)

add_executable(my_benchmark full_example.cpp)
target_link_libraries(my_benchmark PRIVATE nanobench::nanobench)

ankerl::nanobench is a platform-independent microbenchmarking library for C++11, C++14, C++17, and C++20. It is designed to be easy to use, fast, accurate, and robust against outliers.

To use it, include the library in your C++ project and use the provided benchmarking API. A simple benchmark produces a detailed report including nanoseconds per operation, operations per second, error percentage, instructions per operation, cycles per operation, Instructions Per Cycle (IPC), branch prediction behavior, and cache miss rates.

To use nanobench without a package manager, follow these steps:

1. Download nanobench.h from the latest release.
2. Create a dedicated .cpp file (e.g., nanobench.cpp) to compile the bulk of the library once. This prevents long compile times in every translation unit.
3. Compile the library file separately using -O3.

Example compilation command:

g++ -O3 -I../include -c nanobench.cpp

Nanobench uses a Mustache-like template mechanism to export benchmark data into various formats. You can render results using ankerl::nanobench::render() or directly via ankerl::nanobench::Bench::render().

To prevent results from automatically printing to standard output, call bench.output(nullptr) before running benchmarks.