DOCSEARCHbyAlgolia
Submit Docs

Google Benchmark Documentation

repository·main·Indexed 27 days ago

https://github.com/google/benchmark

A C++ library used to benchmark code snippets, providing a structured way to measure performance similar to unit testing frameworks. It includes support for CMake and Bazel integration, Rust bindings, and a suite of assembly tests using LLVM's Filecheck for verifying code generation on x86_64 targets.

Tokens
14K
Snippets
39
Records
89
Agent score
94%

What's inside Google Benchmark

  1. Integrate Google Benchmark with CMake

    main

    You can consume Google Benchmark in a CMake project in three ways:

    1. Installed version: Use find_package(benchmark REQUIRED) and link against benchmark::benchmark or benchmark::benchmark_main.
    2. Source tree (add_subdirectory): Add the repository to your project and use add_subdirectory(path/to/benchmark). It is recommended to disable tests and install rules when doing this.
    3. Object Libraries: If sharing benchmarks through an intermediate CMake target, use an OBJECT library to ensure BENCHMARK registrations are correctly linked into the final executable.

    Link against benchmark::benchmark if you define your own main. Link against benchmark::benchmark_main to use the default entry point.

    # Using an installed version
find_package(benchmark REQUIRED)
target_link_libraries(MyTarget benchmark::benchmark)

# Using add_subdirectory (recommended configuration)
set(BENCHMARK_ENABLE_GTEST_TESTS OFF CACHE BOOL "" FORCE)
set(BENCHMARK_ENABLE_TESTING OFF CACHE BOOL "" FORCE)
set(BENCHMARK_ENABLE_INSTALL OFF CACHE BOOL "" FORCE)
add_subdirectory(third_party/benchmark)
target_link_libraries(MyTarget benchmark::benchmark)

  2. Disable CPU Frequency Scaling to reduce noise

    main

    If you encounter the warning ***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead., you should disable CPU frequency scaling to stabilize performance. On Linux, you can use the cpupower tool to set the governor to performance.

    Note: This must be run as root.

    sudo cpupower frequency-set --governor performance

  3. Reduce run-to-run variance using Random Interleaving

    main

    Random Interleaving is a technique used to lower run-to-run variance in microbenchmarks by randomly interleaving repetitions of one microbenchmark with repetitions from other microbenchmarks within the same test run. On average, this can reduce variance by approximately 40%.

    To enable this feature, use the --benchmark_enable_random_interleaving=true flag. For optimal results, it is recommended to also specify a non-zero repetition count and decrease the per-repetition time.

  4. Link with pthread when building with GCC

    main
    When building the library using GCC, you must link with the pthread library because of how GCC implements std::thread. Failure to do so will result in runtime exceptions rather than linker errors. It is recommended to use the -pthread flag in your linker command instead of -lpthread to avoid potential command-line parameter ordering issues.

  5. Build Google Benchmark Python bindings from source

    main

    To build the Python bindings wheel from source, you must have Bazel installed. Follow these steps to set up a virtual environment and build the wheel:

    1. Clone the repository and enter the directory.
    2. Create and activate a virtual environment using --system-site-packages.
    3. Upgrade pip and build.
    4. Run the build command. The resulting wheel will be stored in the dist directory.
    git clone https://github.com/google/benchmark.git
cd benchmark
# create a virtual environment and activate it
python3 -m venv venv --system-site-packages
source venv/bin/activate  # .\venv\Scripts\Activate.ps1 on Windows

# upgrade Python's system-wide packages
python -m pip install --upgrade pip build
# builds the wheel and stores it in the directory "dist".
python -m build

  6. Configure Visual Studio (2015, 2017, or 2022) for Google Benchmark

    main

    To build with Visual Studio, you must ensure the shlwapi library is linked to support CPUInfo registry reads. You can do this via the IDE settings or by adding a #pragma comment to your code.

    If you are using the static library, you must also define BENCHMARK_STATIC_DEFINE in your preprocessor definitions.

    // Link shlwapi and benchmark libraries via code
#ifdef _WIN32
#pragma comment ( lib, "Shlwapi.lib" )
#ifdef _DEBUG
#pragma comment ( lib, "benchmark.lib" )
#else
#pragma comment ( lib, "benchmark.lib" )
#endif
#endif

// Note: For static linking, ensure BENCHMARK_STATIC_DEFINE 
// is added to [Configuration Properties > C/C++ > Preprocessor > Preprocessor Definitions]

  7. Add Google Benchmark using WORKSPACE

    main

    For projects using the legacy WORKSPACE system, declare Google Benchmark as an external repository using http_archive. You must then load and call benchmark_deps() from @google_benchmark//:bazel/benchmark_deps.bzl to resolve dependencies.

    load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "google_benchmark",
    strip_prefix = "benchmark-<VERSION>",
    urls = ["https://github.com/google/benchmark/archive/refs/tags/v<VERSION>.tar.gz"],
    # Add sha256 for reproducible builds.
)

load("@google_benchmark//:bazel/benchmark_deps.bzl", "benchmark_deps")

benchmark_deps()

  8. Verify Python version requirements for bindings

    main

    If you are installing Google Benchmark Python bindings from PyPI, you must use Python 3.10 or newer (as of v1.9.0).

    If you are building from source, the minimum supported version is Python 3.8, as the underlying bindings generator (nanobind) requires at least Python 3.8.

  10. Use Templated Fixtures

    main

    You can create benchmarks using templated fixtures to test different types with the same logic.

    Standard Templated Fixtures:

    • BENCHMARK_TEMPLATE_F(ClassName, Method, ...): Defines and registers a benchmark using a specific template instantiation.
    • BENCHMARK_TEMPLATE_DEFINE_F(ClassName, Method, ...): Defines a templated benchmark without registering it.

    Method Templates: If you want to define a single method template and instantiate it multiple times for different types, use:

    • BENCHMARK_TEMPLATE_METHOD_F(ClassName, Method): Defines the method template.
    • BENCHMARK_TEMPLATE_INSTANTIATE_F(ClassName, Method, ...): Instantiates and registers the method for specific types.

    Note: When using BENCHMARK_TEMPLATE_METHOD_F, the type Base refers to the instantiated fixture type. Access fixture members using this->.

    template<typename T> 
class MyFixture : public benchmark::Fixture {};

// Defines and registers `IntTest` using `MyFixture<int>`
BENCHMARK_TEMPLATE_F(MyFixture, IntTest, int)(benchmark::State& st) {
   for (auto _ : st) {
     ...
   }
}

// Using method templates
BENCHMARK_TEMPLATE_METHOD_F(MyFixture, Test)(benchmark::State& st) {
   for (auto _ : st) {
     ...
   }
}

// Instantiates and registers for int and double
BENCHMARK_TEMPLATE_INSTANTIATE_F(MyFixture, Test, int)->Threads(2);
BENCHMARK_TEMPLATE_INSTANTIATE_F(MyFixture, Test, double)->Threads(4);

  12. Add Google Benchmark using Bzlmod

    main

    If your Bazel project has Bzlmod enabled, add Google Benchmark to your MODULE.bazel file using bazel_dep.

    To use the default entry point, depend on @google_benchmark//:benchmark_main in your cc_binary or cc_test. When linking against :benchmark_main, your source code should register benchmarks using the BENCHMARK() macro but must not call BENCHMARK_MAIN().

    # In MODULE.bazel
bazel_dep(name = "google_benchmark", version = "<VERSION>")

# In your BUILD file
load("@rules_cc//cc:defs.bzl", "cc_binary")

cc_binary(
    name = "string_benchmark",
    srcs = ["string_benchmark.cc"],
    deps = ["@google_benchmark//:benchmark_main"],
)