ForkUnion

ForkUnion is a NUMA-aware fork-join thread-pool designed for high-performance parallel loops (similar to #pragma omp parallel for). Unlike traditional task-queue runtimes, it is optimized for low-latency dispatch in high-core-count environments.

Key Characteristics:

• Performance: Designed to minimize the 'fork-join tax'. It is significantly faster than OpenMP, Rayon, and Taskflow for parallel-for dispatches.
• Zero Overhead on Hot Path: Makes zero heap allocations, zero system calls, and zero CAS (Compare-And-Swap) operations during execution. It avoids cache-line false-sharing.
• Hardware-Level Parking: Uses hardware address monitors (e.g., x86 UMONITOR/UMWAIT, Arm WFET, RISC-V Zawrs) to allow idle workers to light-sleep on specific cache-lines, waking instantly upon a write without hot spinning or kernel futexes.
• Scalability: Maintains flat dispatch latency even into hundreds of cores by using private fetch_add cursors to avoid interconnect thrashing on multi-socket systems.
• Portability: Runs on Linux, FreeBSD, Windows, macOS, Android, and iOS. Supports asymmetric compute and memory topologies.

ForkUnion models hardware using two independent axes to handle non-uniformity:

1. Compute Domain: A bindable group of cores sharing a Quality-of-Service (QoS) class and locality. This is what a pool spawns onto.
2. Memory Domain: A bank of memory with its own capacity and access cost (e.g., HBM, DDR, CXL). This is what the allocator targets.

Key Concepts:

• Levels: Dense ordinal groupings of domains with similar performance.
  • Compute levels grow with performance (higher is faster).
  • Memory levels grow with distance (lower is faster/closer).
• Topology vs. Fabric:
  • Topology holds what the platform declares (domains, cores, QoS, volumes).
  • Fabric holds what ForkUnion observes (per-edge latencies, bandwidths, and distances).
• Magnitudes:
  • compute_capacity_in: Kernel's rating of a core (normalized to 1024 for the fastest core; 0 on x86/Windows/Apple).
  • compute_cache_bytes_in: Deepest private cache size.
  • memory_distance: Relative cost of reaching memory (10 for local, higher for remote).
  • memory_bandwidth / memory_latency: Observed saturated read bandwidth and dependent-load latency.

ForkUnion is designed for high-performance data parallelism and tight parallel loops, avoiding nested parallelism, exception handling, or future/promises.

Core Operations

• try_spawn: Initializes worker threads.
• for_threads: Launches a blocking callback on all threads.

Index-Addressable Task APIs

• for_n: Distributes individual, evenly-sized tasks (static scheduling).
• for_n_dynamic: Distributes individual, unevenly-sized tasks (dynamic scheduling/work stealing).
• for_slices: Distributes slices of evenly-sized tasks.

Flow Control and Tuning

• sleep(microseconds): For longer naps.
• terminate: Kills threads before the destructor is called.
• unsafe_for_threads: Broadcasts a callback without blocking; returns a generation token.
• unsafe_join: Blocks until a broadcasted generation is complete.
• is_complete: Polls a generation token for completion without blocking.

ForkUnion's thread-pool relies on a minimal set of atomic variables to manage lifecycle and task distribution. Understanding these helps in reasoning about the library's safety and performance.

Key Safety Guarantees:

• Thread Count: The number of threads can only be changed by calling terminate and then try_spawn. These operations are mutually exclusive with running tasks, ensured by the stop variable.
• Job Submission: New tasks are submitted from a single thread that updates the number of parts for each new fork. Workers remain asleep on old fork_generation values until the increment occurs, ensuring they only access new job pointers when safe.

ForkUnion provides a parallel iterator API in Rust via the prelude. Unlike Rayon, these iterators allow explicit control over the thread pool and scheduling strategy.

Static Scheduling (Default): Most efficient for statically shaped workloads. Dynamic Work-Stealing: Use .with_schedule(&mut pool, DynamicScheduler) for workloads with unpredictable task durations.

Common Operations:

• map, filter, zip: Standard iterator adaptors.
• sum, count: Parallel reductions.
• reduce: Custom parallel reductions.
• find_first, find_last, find_any: Short-circuiting searches.
• try_for_each, try_fold_with_scratch: Fallible parallel operations that stop on the first error.

use forkunion as fu;
use forkunion::prelude::*;

let topology = fu::Topology::new().expect("Failed to detect hardware topology");
let mut pool = fu::spawn(&topology, 4);
let mut data: Vec<usize> = (0..1000).collect();

// Static scheduling
(&data[..])
    .into_par_iter()
    .with_pool(&mut pool)
    .for_each(|value| {
        println!("Value: {}", value);
    });

// Dynamic work-stealing
(&mut data[..])
    .into_par_iter()
    .with_schedule(&mut pool, DynamicScheduler)
    .for_each(|value| {
        *value *= 2;
    });