RingMPSC

RingMPSC is designed for extreme throughput by being memory-bandwidth bound rather than synchronization bound. It achieves near-linear scaling up to 6 producers (95%+ efficiency) and maintains ~80% efficiency at 8 producers on modern hardware (e.g., Zen 3).

Throughput Benchmarks (8P1C)

Key Performance Drivers

• Ring Decomposition: Each producer has its own ring, allowing producers to write into their own L2 cache without contention.
• Batch Amortization: Using large batches (e.g., 32K) amortizes the cost of atomic operations (reserve, commit, consume).
• Cache Isolation: Hot variables are separated by 128 bytes to prevent false sharing and assist prefetcher isolation.
• Memory Bandwidth: Throughput is primarily limited by the L2/L3 cache bandwidth and memory controller coherency traffic rather than lock contention.

The BufferPool manages memory slabs using a lock-free free list. It utilizes Compare-And-Swap (CAS) with ABA-safe tagged pointers to ensure correctness during high-concurrency allocation and deallocation.

Implementation Details

• Tagged Pointers: The free_head is a u64 where the upper 32 bits are a tag (counter) and the lower 32 bits are the index of the free slab.
• ABA Prevention: Every time a slab is allocated or freed, the 32-bit tag is incremented. This ensures that even if a slab index is reused, the CAS operation will fail if the tag does not match, preventing stale updates.

Allocation and Freeing Logic

• alloc(): Loads the current free_head, extracts the index and tag, finds the next_free index from the metadata, and attempts to CAS the free_head to a new value containing the incremented tag and the next index.
• free(handle): Loads the free_head, sets the metadata of the slab being freed to point to the current head, and attempts to CAS the free_head to a new value containing the incremented tag and the current handle.

```rust
// free_head structure: (tag:u32 << 32 | index:u32)

// alloc() logic
loop {
    head = free_head.load(.acquire)
    idx = head & 0xFFFFFFFF
    tag = head >> 32
    next = meta[idx].next_free
    new_head = ((tag + 1) << 32) | next
    if cmpxchg(free_head, head, new_head): return idx
}

// free(handle) logic
loop {
    head = free_head.load(.acquire)
    tag = head >> 32
    meta[handle].next_free = head & 0xFFFFFFFF
    new_head = ((tag + 1) << 32) | handle
    if cmpxchg(free_head, head, new_head): return
}

RingMPSC is a Multi-Producer Single-Consumer (MPSC) channel that achieves high throughput by using ring decomposition. Instead of all producers contending for a single shared tail pointer (which causes cache-line bouncing and CAS failures), RingMPSC assigns each producer its own dedicated Single-Producer Single-Consumer (SPSC) ring buffer.

The Workflow:

1. Producers: Each producer operates on its private SPSC ring with zero contention from other producers.
2. Consumer: The consumer polls all active rings in a round-robin fashion, draining each one in turn.

Trade-off: This design trades memory for throughput. Memory usage scales as O(P × capacity), where P is the number of producers. For example, 8 producers using 64K-slot rings of 4-byte elements requires approximately 2MB of memory.

Producer 0 ──► [Ring 0] ──┐
Producer 1 ──► [Ring 1] ──┼──► Consumer (polls all rings)
Producer 2 ──► [Ring 2] ──┤
    ...                   │
Producer N ──► [Ring N] ──┘

To avoid the performance penalty of copying large messages (typically > 64 bytes) into and out of the ring, use the Pointer-Ring Pattern. Instead of storing the payload in the ring, the ring stores a Handle (u32) that points to a slab in a pre-allocated BufferPool.

Ownership Model

Ownership is transferred explicitly via the ring, ensuring no two threads access the same slab simultaneously:

1. pool.alloc(): The producer gains exclusive ownership of a slab.
2. ring.send(handle): Ownership transfers from the producer to the ring.
3. ring.recv(): Ownership transfers from the ring to the consumer.
4. pool.free(handle): The consumer returns the slab to the pool.

This pattern achieves zero-copy throughput by allowing the producer to write directly into the pool slab and the consumer to read from that same physical memory.

// Conceptual workflow
let handle = pool.alloc();
// ... producer writes data directly into the slab at handle ...
ring.send(handle);

let received_handle = ring.recv();
// ... consumer reads data directly from the slab at received_handle ...
pool.free(received_handle);

The primitives.BufferPool is a lock-free slab allocator designed for zero-copy transmission of large payloads. Instead of passing large data through a ring buffer, you pass a Handle (u32) that points to a fixed-size slab in the pool.

Key Workflow (Pointer-Ring Pattern):

1. Producer: alloc() a handle $\rightarrow$ getWritable() to write data $\rightarrow$ setLen() to record payload size $\rightarrow$ send() the handle through a ring.
2. Consumer: recvOne() the handle $\rightarrow$ get() to read the payload $\rightarrow$ free() the handle to return it to the pool.

Thread Safety: alloc() and free() are lock-free and ABA-safe using tagged pointers. However, get(), getWritable(), and setLen() are not thread-safe on the same handle; ownership must be transferred (e.g., via a ring) to ensure exclusive access.

const Pool = ringmpsc.primitives.BufferPool(4096, 256); // 4KB slabs, 256 of them
var pool: Pool = .{};
var ring = ringmpsc.primitives.Ring(ringmpsc.primitives.Handle, .{ .ring_bits = 10 }){};

// Producer:
const handle = pool.alloc() orelse return error.PoolExhausted;
const buf = pool.getWritable(handle);
@memcpy(buf[0..data.len], data);         // write directly to pool (zero copy)
pool.setLen(handle, @intCast(data.len));
_ = ring.send(&[_]Handle{handle});        // 4 bytes through ring, not 4096

// Consumer:
const h = ring.recvOne() orelse continue;
const payload = pool.get(h);              // read from same address (zero copy)
process(payload);
pool.free(h);                             // return to pool

To use RingMPSC effectively, familiarize yourself with the following core concepts:

• MPSC (Multi-Producer Single-Consumer): A pattern where many threads send messages and exactly one thread receives them.
• SPSC (Single-Producer Single-Consumer): A pattern with one sender and one receiver; this is the fastest possible configuration.
• Ring decomposition: An optimization where each producer is given its own dedicated SPSC ring instead of all producers sharing a single queue.
• Batch consumption: A technique for processing many messages with a single atomic head update to amortize synchronization costs.
• False sharing: A performance pitfall where two threads' data reside on the same cache line, causing spurious invalidations.
• Acquire/Release: Memory ordering used to ensure writes before a release-store are visible after an acquire-load.
• CAS (Compare-And-Swap): An atomic read-modify-write operation that may fail under high contention.
• Futex (Fast userspace mutex): An OS-level sleep/wake mechanism with zero overhead when there is no contention.