DOCSEARCHbyAlgolia
Submit Docs

Highway SIMD Library

repository·master·Indexed 24 days ago

https://github.com/google/highway

A C++17 library providing portable SIMD/vector intrinsics for high-performance, data-parallel programming across diverse CPU architectures. It supports both static and dynamic dispatch to target multiple instruction sets with a single codebase. The library includes specialized components such as VQSort for vectorized quicksort and an experimental Unroller for automating URHT optimizations.

Tokens
26.6K
Snippets
21
Records
144
Agent score
85%

What's inside Highway

  1. Overview of Highway SIMD library

    master

    Highway is a C++ library designed to provide portable SIMD (Single Instruction, Multiple Data) / vector intrinsics. It allows developers to write high-performance code that can target multiple CPU architectures (including those with scalable vectors) using a single codebase.

    Key features include:

    • Predictable Performance: Maps well to CPU instructions without relying heavily on complex compiler autovectorization.
    • Portability: Supports seven architectures and requires C++17.
    • Flexible Dispatch: Supports both HWY_STATIC_DISPATCH (targeting a single instruction set via compiler flags) and HWY_DYNAMIC_DISPATCH (choosing the best available instruction set at runtime).

  2. Compare Highway with std::simd

    master

    Highway is recommended over std::simd for high-performance SIMD applications due to several technical advantages:

    • Scalable Vector Support: Unlike std::simd, which uses constexpr-sized class wrappers incompatible with Arm SVE and RISC-V V, Highway is length-agnostic. It uses ScalableTag<T> to adapt to whatever hardware vector length is present, using built-in types directly on SVE/RVV to avoid class wrapper issues.
    • Runtime Dispatch: std::simd only compiles a single codepath determined by compiler flags, which can lead to crashes on older CPUs or underutilization of newer instructions (like AVX-512). Highway is designed for runtime dispatch, allowing a single binary to automatically choose the best implementation for the current CPU at runtime.
    • Extensive Operation Set: Highway supports over 400 operations, including critical missing std::simd ops like byte-level table lookups (PSHUFB), SAD (sum of absolute differences), saturating arithmetic, integer/BF16 multiply-accumulate, and cryptographic primitives (AES, carryless multiply).
    • Availability: Highway is available via major package managers and supports CMake, Bazel, and Meson, whereas std::simd availability depends on C++26 support and standard library implementation quality.

  3. Understand Highway Target Clusters

    master

    Highway uses 'clusters' of related instruction set features rather than individual flags to manage compile time and code size. For example, HWY_AVX2 represents a group of features. To use a specific target, you must pass the corresponding compiler flags (e.g., -mavx2 for x86) to the compiler.

    Common target clusters include:

    • x86: SSE2/SSSE3/SSE4/AVX2/AVX3/AVX3_DL/AVX3_ZEN4/AVX3_SPR
    • Arm: NEON_WITHOUT_AES/NEON/NEON_BF16/SVE/SVE2/SVE_256/SVE2_128
    • RISC-V: RVV
    • WebAssembly: WASM/WASM_EMU256
    • Power: PPC8/PPC9/PPC10
    • IBM Z: Z14/Z15

  4. Understand Highway's design philosophy and performance goals

    master

    Highway is designed for performance portability, aiming to achieve performance within 10-20% of hand-written assembly while maintaining high readability and maintainability.

    Key principles:

    • Performance Portability: The API provides a carefully chosen set of vector types and operations that are efficient across all target platforms (e.g., Armv8, PPC8, x86).
    • Width-Agnosticism: The library favors width-agnostic SIMD (where vector width is determined at runtime/by the library) over fixed sizes to ensure future-proofing for hardware like Arm SVE or RISC-V V.
    • Explicit Costs: To follow the "pay only for what you use" principle, Highway makes operation costs visible and predictable. For example, conversions between integer and float types must be explicit to avoid hidden performance penalties.
    • Runtime Dispatch: Highway supports efficient runtime dispatch, allowing a single binary to contain multiple instruction set paths (e.g., SSE, AVX2, AVX-512) and choose the best one at runtime. To minimize overhead, it is recommended to hoist dispatch to higher layers rather than checking inside every low-level function.

  5. Strip-mining loops for vectorization

    master

    When vectorizing a loop where the iteration count (count) does not evenly divide the vector width N = Lanes(d), you must use a 'strip-mining' strategy to handle the remainder.

    Assume a loop body defined as: template<bool partial, class D> void LoopBody(D d, size_t index, size_t max_n).

    Strategies for loop vectorization:

    1. Padding (Preferred): Ensure all inputs/outputs are padded so the loop can always process full vectors.

      for (size_t i = 0; i < count; i += N) LoopBody<false>(d, i, 0);

    2. Idempotent Overlap: Process whole vectors and include previously processed elements in the last vector. This is preferred if count >= N and LoopBody is idempotent.

      for (size_t i = 0; i < count; i += N) LoopBody<false>(d, HWY_MIN(i, count - N), 0);

    3. Transform Functions: Use Transform* functions from hwy/contrib/algo/transform-inl.h. This handles the loop and remainder automatically via a lambda or functor.

      Transform1(d, x, n, y, [](auto d, const auto v, const auto v1) HWY_ATTR {
  return MulAdd(Set(d, alpha), v, v1);
});

    4. Scalar Remainder Loop: Process whole vectors until the remainder is less than N, then use a standard scalar loop.

      size_t i = 0;
for (; i + N <= count; i += N) LoopBody<false>(d, i, 0);
for (; i < count; ++i) LoopBody<false>(CappedTag<T, 1>(), i, 0);

    5. Masked Remainder (Best for non-padded data): Process whole vectors, then use a single call to a modified LoopBody with masking for the remaining elements. This is safe only if #if !HWY_MEM_OPS_MIGHT_FAULT is true.

      size_t i = 0;
for (; i + N <= count; i += N) {
  LoopBody<false>(d, i, 0);
}
if (i < count) {
  LoopBody<true>(d, i, count - i);
}

      Inside LoopBody<true>, use BlendedStore(v, FirstN(d, num_remaining), d, pointer); or MaskedLoad(FirstN(d, num_remaining), d, pointer); to handle the partial vector.

    // Example: SAXPY using Transform1
Transform1(d, x, n, y, [](auto d, const auto v, const auto v1) HWY_ATTR {
  return MulAdd(Set(d, alpha), v, v1);
});

  6. Use Static vs Dynamic Dispatch in Highway

    master

    Top-level functions should receive pointers to arrays rather than target-specific vector types.

    Static Dispatch

    HWY_TARGET is the best available target among HWY_BASELINE_TARGETS. Functions inside HWY_NAMESPACE can be called using HWY_STATIC_DISPATCH(func)(args) within the same module. To call them from other modules, wrap them in a regular function declared in a header.

    Dynamic Dispatch

    A table of function pointers is generated via the HWY_EXPORT macro. Use HWY_DYNAMIC_DISPATCH(func)(args) to call the best function pointer for the current CPU.

    Note: The first invocation of HWY_DYNAMIC_* involves CPU detection overhead. To prevent this, call hwy::GetChosenTarget().Update(hwy::SupportedTargets()); before any dynamic dispatch calls.

  7. Use per-target include guards in -inl.h files

    master

    Files ending in -inl.h contain inlined function templates. To support multiple compilation passes required for dynamic dispatch, use a 'per-target include guard' pattern. This allows the include guard to be 'reset' when the translation unit is re-included for the next target.

    #if defined(HWY_PATH_NAME_INL_H_) == defined(HWY_TARGET_TOGGLE)
#ifdef HWY_PATH_NAME_INL_H_
#undef HWY_PATH_NAME_INL_H_
#else
#define HWY_PATH_NAME_INL_H_
#endif
// contents to include once per target
#endif  // HWY_PATH_NAME_INL_H_

  8. Make code safe for ASan and MSan

    master

    When dealing with array remainders that are not divisible by the vector length, using LoadU or MaskedLoad with FirstN(d, remaining_lanes) can trigger page faults or AddressSanitizer (ASan) errors.

    Recommended Approach: Use hwy/contrib/algo/transform-inl.h. Instead of manually writing loops and remainder handling, define a templated lambda function for a single loop iteration. Use the Generate or Transform* functions to handle the remainder logic automatically and safely.

  9. Install Highway via CMake

    master

    Highway uses CMake for building. On Debian-based systems, ensure cmake is installed. To build Highway as a shared or static library (controlled by BUILD_SHARED_LIBS), use the standard CMake workflow.

    To avoid downloading googletest automatically, set HWY_SYSTEM_GTEST=ON and install libgtest-dev separately.

    # Install CMake
sudo apt install cmake

# Optional: Install gtest separately to avoid automatic download
sudo apt install libgtest-dev

# Standard CMake build workflow
mkdir -p build && cd build
cmake ..
make -j && make test

  10. Optimize memory access with aligned loads and stores

    master
    While modern CPUs (like Intel Haswell+) handle unaligned loads efficiently, unaligned loads can consume extra load ports (splitting one load into two), which may slow down low-arithmetic-intensity algorithms. Unaligned stores are also typically more expensive. For maximum efficiency, use aligned loads and stores where possible. Note that specialized memory operations like CompressStore or BlendedStore are not specialized for aligned pointers to avoid doubling the number of memory operations.

  11. Choose between Static and Dynamic Dispatch

    master

    Highway allows you to control how SIMD instructions are selected at runtime or compile time:

    1. Static Dispatch: Use HWY_STATIC_DISPATCH to target a specific instruction set. This is simpler but requires the application to be compiled with specific flags (e.g., -m flags) to enable the desired instruction set.
    2. Dynamic Dispatch: Use HWY_DYNAMIC_DISPATCH to allow the application to detect and use the best available instruction set on the host machine at runtime. This provides maximum flexibility for heterogeneous environments.

  12. Start using Highway with boilerplate examples

    master

    The easiest way to start is to copy an existing Highway example file, such as hwy/examples/benchmark.cc or skeleton.cc. This ensures correct namespaces and include order.

    To implement your own logic:

    1. Insert your code into RunBenchmarks (for benchmark.cc) or FloorLog2 (for skeleton.cc).
    2. For initial development, you can write standard C++. The compiler may autovectorize this code if it is straightforward.
    3. To implement vectorized logic, wrap your code in a target check:
      • Use #if HWY_TARGET == HWY_SCALAR || HWY_TARGET == HWY_EMU128 for the scalar/emulated fallback.
      • Use the #else branch for your vectorized implementation using Highway intrinsics.

    If you create a test by copying a file from hwy/tests/, Highway will automatically run your test across all supported targets.