XNNPACK Documentation

XNNPACK is a highly optimized library for neural network inference designed for ARM, x86, WebAssembly, and RISC-V platforms.

Note for Users: XNNPACK is not intended for direct use by deep learning practitioners or researchers. It is designed to provide low-level performance primitives to accelerate high-level machine learning frameworks.

Common frameworks that use XNNPACK include:

• TensorFlow Lite
• TensorFlow.js
• PyTorch
• ONNX Runtime
• ExecuTorch
• MediaPipe

XNNPACK uses a macro-based enumeration pattern to avoid duplicating code for tests, benchmarks, and function declarations across different microkernels. To iterate over all microkernels of a specific type, you must #define a macro (e.g., XNN_UKERNEL) that specifies how to handle the microkernel's parameters, then #include the corresponding .inc header file.

It is recommended to #undef the macro immediately after the #include to prevent macro name collisions in other parts of your code.

#define XNN_UKERNEL(arch_flags, fn_name, batch_tile, vector_tile, datatype) \
    printf("%s %d\n", #fn_name, batch_tile);
#include "src/f32-vtanh/f32-vtanh.inc"
#undef XNN_UKERNEL

Depthwise convolution microkernels in XNNPACK are specialized low-level implementations designed to produce one row of output per call. They are located in src/*-dwconv directories (e.g., src/f32-dwconv).

Key Concepts

• Channel Tile: The number of channels the microkernel processes in a single main loop iteration.
• Kernel Tile: The number of weights (kernel elements) read in each iteration. This may be larger than the actual number of kernel elements.
• Main Loop: Processes channel_tile outputs per iteration, reading channel_tile biases, channel_tile * kernel_tile inputs, and channel_tile * kernel_tile weights.
• Remainder Loop: Handles the remaining channels when the total number of channels is not a multiple of the channel_tile.

Building for Android requires the Android NDK. You must set the ANDROID_NDK environment variable to the path where your NDK is unpacked. Use the provided scripts/build-android-armv7.sh script for 32-bit Arm builds.

# Set ANDROID_NDK to wherever the NDK is unpacked
export ANDROID_NDK=$HOME/bin/android-ndk-r27c

# Build for 32-bit Arm
./scripts/build-android-armv7.sh

# Push the compiled benchmarks onto an ADB-connected device
adb push ./build/android/armeabi-v7a/bench /data/local/tmp

Use the scripts/build-local.sh script to automatically configure and run CMake for your host CPU and operating system. Build artifacts are placed in build/local.

To run the test suite after building, use ctest from the build directory. You can use --parallel $(nproc) to run tests in parallel using all available CPU cores.

# In the XNNPACK root directory
scripts/build-local.sh

# Build artifacts will be created in build/local
cd build/local

# Run the test suite
ctest --output-on-failure --parallel $(nproc)

You can pass standard CMake arguments to scripts/build-local.sh to specify different C and C++ compilers (e.g., using clang instead of the default).

scripts/build-local.sh -DCMAKE_CXX_COMPILER=clang++-19 -DCMAKE_C_COMPILER=clang-19
cd build/local
ctest --output-on-failure --parallel $(nproc)

GN support is experimental and uses Chromium's depot_tools to manage dependencies.

1. Setup Dependencies

Create a .gclient file in your XNNPACK directory to manage the source and dependencies via gclient sync.

2. Configure the Build

Run gn args out/Default to open a configuration editor. Common arguments include:

• target_cpu: "x64", "x86", or "arm64".
• is_debug: boolean.
• dcheck_always_on: boolean (enables assertions).
• symbol_level: integer (e.g., 1 to retain some symbol information).
• xnnpack_enable_avx512: boolean.

To regenerate files after editing args.gn, run gn gen out/Default.

3. Build and Test

Use autoninja to build and a helper script to run tests.

To update a dependency located in the third_party directory, follow these steps:

1. Update the //DEPS file with the desired revision of the library.
2. Run gclient sync to fetch the updated dependency.
3. Build the project for a variety of platforms to ensure compatibility.

XNNPACK supports Bazel for building.

Important Notes:

• Cross-compilation to another OS or architecture is not supported on recent Bazel versions.
• Do not use the :all meta-target; use explicit targets like //bench/... or //test/....
• Use the -c flag to control optimization levels:
  • -c opt: Recommended for production (enables -O2, no assertions/debug info).
  • -c fastbuild: For development (no optimization, minimal debug info, assertions enabled).
  • -c dbg: For debugging (debug information enabled, no optimization).

To use an alternate compiler, set the CC and CXX environment variables.

To build XNNPACK, your environment must meet the following minimum requirements:

• C11
• C++17
• Python 3