libfacedetection

The project follows a specific organizational pattern for SIMD to keep unsafe code auditable and safe:

• kernels::mod: Handles platform detection and fallback logic.
• kernels::x86: Contains the actual AVX2 intrinsics.
• kernels::scalar: Provides a readable, cross-platform reference implementation used as a fallback.
• Upper Layers: layers and network modules interact with the kernels through safe abstractions and do not touch intrinsics directly.

Each AVX2 entry point assumes specific shape preconditions (e.g., channel padding, specific output channel widths).

Efficient memory management in the Highway implementation relies on reusing large temporary buffers (scratchpads) and output blobs to minimize allocator churn.

• HwNetworkWorkspace: A persistent container that owns reusable scratch blobs for the backbone and FPN/head stages, as well as reusable backbone outputs for full-network calls.
• HwBlob::ResizeForOverwrite: A method used to resize a blob while preserving its vector capacity. This is particularly useful when reusing shapes, as it avoids the overhead of clearing large outputs that will be completely overwritten by the next operation.
• Steady-state reuse: For optimal performance in a loop (e.g., video processing), use thread_local instances of the input blob, network workspace, raw head outputs, and decoded outputs to ensure memory is allocated once and reused.

The library does not use a single SIMD implementation for all operations. Instead, it employs a Hybrid Ceiling strategy, selecting the fastest backend based on the operation type to balance cross-platform abstraction (Highway) with raw performance (Intrinsics).

Current selection logic:

• Pointwise 1x1: Uses Highway packed pointwise.
• Depthwise 3x3: Uses intrinsics depthwise.
• Maxpool 2x2: Uses intrinsics maxpool.
• Vector Add: Uses Highway.

Note that for pointwise operations, a strategy selector is used to decide between packed and primitive implementations based on channel counts: packed when channels >= 16 and out_channels >= 16; primitive otherwise.

The Blob type is the core tensor abstraction used for HWC (Height-Width-Channel) f32 storage. It is designed to be performance-oriented and AVX2-friendly.

Key Features:

• Padded Storage: Uses a channel_step to provide default channel padding (8 floats) to ensure deterministic SIMD reads.
• Memory Management:
  • resize: Clears allocation.
  • resize_for_overwrite: Reuses existing capacity/shape to avoid new allocations during workspace reuse.
• Views: Supports both immutable and mutable view types for kernel operations.

Kernels in this library (e.g., relu_in_place, add_to, pointwise_1x1_to) write directly into caller-owned BlobViewMut outputs to avoid internal allocations.

The Highway-optimized (hw) detection pipeline is composed of several distinct stages. Understanding these helps in profiling and identifying performance bottlenecks:

1. Image Transform: Converts input to 32-channel initial feature blobs.
2. Backbone: The primary computational load (e.g., conv_head, conv0 through conv5).
3. FPN + Raw Heads: Fused layers (using UpsampleX2AddHw) that add lateral connections to the backbone outputs.
4. Decode + Concat: Post-processing tensors using helpers like MeshgridHw, BboxDecodeHw, and SigmoidHw.
5. NMS: Non-Maximum Suppression to filter overlapping detections.

The Highway implementation uses a PointwiseStrategy to choose the most efficient kernel based on the layer dimensions. This prevents the overhead of 'packed' kernels on very small layers where padding and tail handling would degrade performance.

Selection Logic:

• packed strategy: Used when channels >= 16 AND out_channels >= 16. This is optimized for backbone and neck layers (e.g., 16/32/64-output layers).
• primitive strategy: Used otherwise. This is optimized for tiny detection heads (e.g., 1/4/10-output layers).

This strategy is encapsulated in PointwisePlan objects, which are created once during model initialization (via HwFilter) to avoid per-call overhead.

To minimize runtime overhead, the library moves pointwise packing to the model loading stage.

Filter and Model Structure:

• Filter: Owns padded weights and compact biases. It records whether with_relu is enabled.
• PointwisePlan: A persistent plan created during Filter::load to handle pointwise convolutions efficiently.
  • Primitive: Used for tiny heads or layers with small output counts.
  • Packed: Used when channels >= 16 && out_channels >= 16. It stores weights as [input_channel][padded_output_channel] with 8-lane packing to match AVX2 baselines.

Runtime Dispatch: During the forward pass, the Model dispatches convolution through these pre-computed plans, ensuring that large pointwise layers use optimized paths without changing kernel signatures.

The detection process follows a specific sequence to ensure correctness before optimization. The pipeline includes:

1. Image Transform: Pre-processing the input image.
2. Neural Network Inference: Executing the backbone, FPN, and various heads (cls/reg/kps/obj).
3. Post-processing:
   • Meshgrid generation
   • Bbox/Keypoint decoding
   • Sigmoid activation
   • Confidence filtering
   • Stable descending score sorting
   • Non-Maximum Suppression (NMS)
   • Keep-top-k selection
4. Output Generation: Packing results into either Rust-structured Face objects or C-compatible result buffers.

The hw implementation is structured to separate high-level model flow from low-level SIMD kernels. The execution flow follows this hierarchy:

facedetect_hw_cnn $\rightarrow$ hw_objectdetect_cnn $\rightarrow$ hw convolution flow $\rightarrow$ hw kernels.

Available Primitive Kernels:

• hw_dot_product(const float* a, const float* b, int n)
• hw_mul_add(const float* a, const float* b, float* acc, int n)
• hw_add(const float* a, const float* b, float* out, int n)
• hw_relu(float* data, int n)
• hw_maxpool_2x2s2(...)
• hw_pointwise_1x1(...)
• hw_depthwise_3x3(...)
• hw_element_add(...)