nvidia/stdexec

repository·main·Indexed 25 days ago

https://github.com/nvidia/stdexec

A reference implementation of the C++26 std::execution (P2300) model. It provides a framework for asynchronous and parallel programming using composable, lazy sender pipelines with structured concurrency guarantees. The library is organized into namespaces for standard components (stdexec), generic extensions (exec), and NVIDIA-specific schedulers (nvexec), including support for GPU offload via the NVIDIA HPC SDK.

Tokens
23.9K
Snippets
57
Records
88
Agent score
82%

What's inside stdexec

  1. What is the Sender abstraction in stdexec?

    main

    The Sender abstraction is a foundational model for asynchronous programming in C++, serving as the reference implementation for the std::execution framework proposed for C++26.

    Unlike legacy mechanisms (like std::async, futures, or callbacks), the Sender abstraction provides a compositional model that separates the what (the computation) from the where (the execution context/scheduler).

    Key benefits include:

    • Unified Model: Works across compute, I/O, networking, and UI.
    • Zero-Overhead Composition: Uses compile-time plumbing to avoid runtime allocations or reference counting.
    • Structured Control: Built-in support for cancellation and error handling.
    • Coroutine Integration: Senders can be directly co_await-ed within coroutines.
    • Customization: Allows plugging in custom schedulers, adaptors, allocators, and stop tokens.
  2. Implement custom senders, receivers, or schedulers using CPOs

    main

    If you are writing a new sender, receiver, or scheduler, you must implement the Core Customization Points (CPOs) to participate in the protocol. Most users only interact with these via adaptors, but implementers must support them:

    Sender-side CPOs

    • connect: Describes how to connect a sender to a receiver.
    • get_completion_signatures: A function template used to expose the computation's signatures.
    • get_env: Describes how the sender exposes its environment.

    Operation-state-side CPOs

    • start: The trigger that turns a connected sender into a running operation.

    Receiver-side CPOs

    • set_value: Delivers a successful completion to the receiver.
    • set_error: Delivers an error completion to the receiver.
    • set_stopped: Delivers a stopped/cancelled completion to the receiver.
    • get_env: Allows the operation state to query the receiver's environment.
  3. Use `stdexec::on` for round-trip scheduling

    main

    The stdexec::on adaptor is used for round-trip scheduling. It runs work on a target scheduler and then transfers execution back to the original scheduler that started the operation.

    There are two forms:

    1. on(sched, sndr): Runs the entire sndr on sched and then returns to the starting scheduler. Use this when downstream code needs to run on the caller's scheduler again. This differs from starts_on, which stays on the new scheduler.

    2. on(sndr, sched, closure) (or the pipe form sndr | on(sched, closure))**: This is the "side trip" pattern. The predecessor runs on its own scheduler, the execution hops to sched for the closure, and then hops back when the closure completes. Use this for small, well-defined chunks of work (like a GPU kernel or a blocking syscall) that need a different scheduler while the rest of the pipeline remains unchanged.

    // Form 1: Run entirety of sndr on sched and return
    auto sched = stdexec::get_parallel_scheduler();
    auto sndr  = stdexec::on(sched, stdexec::just(21)
                                      | stdexec::then([](int x){ return x*2; }));
    
    // Form 2: Side trip pattern
    auto gpu = stdexec::get_parallel_scheduler();  // pretend: GPU
    auto sndr = stdexec::just(21)
          | stdexec::on(gpu, stdexec::then([](int x) { return x * 2; }));
    // The then() inside runs on `gpu`, but sync_wait() sees the result on its own context.
  4. Understand the stdexec foundational concepts

    main

    The stdexec API is built on three layers of concepts that define how asynchronous computations are described, consumed, and executed:

    1. Sender side: Describes the computation. A sender is a value that describes an async computation without executing it yet.
      • Key concepts: sender, sender_in, sender_to.
    2. Receiver / operation-state side: Describes the consumer and the running task. This layer handles the destination of completion signals and the actual running operation.
      • Key concepts: receiver, receiver_of, operation_state.
    3. Context side: Describes execution resources and lifetime tracking. This layer manages how work is dispatched and how its lifetime is tracked.
      • Key concepts: scheduler, scope_token, scope_association.
  5. Core Concepts for Sender Model Developers

    main

    Developers extending the Sender model (writing new algorithms, schedulers, or adapting other models) must understand the interaction between five core components:

    1. Scheduler: Produces a Sender via schedule(scheduler).
    2. Sender: Produces an Operation State via connect(sender, receiver).
    3. Receiver: Consumes results via completion signals (set_value, set_error, or set_stopped).
    4. Environment: A key/value store (queryable via get_env) used to pass contextual information like stop tokens, allocators, or schedulers to receivers.
    5. Operation State: Represents the in-progress computation. It is produced by connecting a sender to a receiver and must be explicitly started via .start().
  6. How domains allow pipeline rewriting

    main

    In stdexec, domains are used to allow schedulers or algorithms to rewrite sender expressions in a pipeline during the connect phase. This is essential for specialized execution contexts, such as GPU schedulers that need to transform standard algorithms (like stdexec::then) into CUDA kernel launches, or tracing schedulers that inject span-recording code.

    A domain is a tag type that implements a transform_sender static method. The framework consults this method at connect time, allowing it to replace the original sender with a rewritten version.

    There are two primary customization paths:

    1. Tag-type customization: Used by sender adaptor authors. You define a static transform_sender on the adaptor's tag type (e.g., foo_t). This makes the rewrite universal regardless of the active domain.
    2. Domain-level customization: Used by scheduler authors. You define a custom domain type and publish it through the scheduler's environment. The framework then routes every sender in the pipeline through your domain's transform_sender first.
    struct my_domain {
          template <class OpTag, class Sndr, class Env>
          static auto transform_sender(OpTag, Sndr&& sndr, Env const&)
            /* -> some-new-sender-or-the-same-sender */;
        };
  7. Recover from errors with `stdexec::upon_error` and `stdexec::let_error`

    main

    These adaptors allow you to handle errors in the error channel of a sender.

    stdexec::upon_error (Synchronous Recovery)

    Use this when your recovery logic returns a value. It consumes the error channel; the resulting sender will no longer complete via set_error (unless the recovery callable itself throws).

    stdexec::let_error (Asynchronous Recovery)

    Use this when your recovery logic (e.g., a retry or fallback fetch) returns a sender. This allows the recovery step to be an asynchronous operation.

    Note: If the predecessor succeeds or is cancelled, these adaptors are no-ops and the completion is forwarded unchanged.

    // upon_error example (returns a value)
    auto sndr = stdexec::just_error(std::error_code{ENOENT, std::system_category()})
                  | stdexec::upon_error([](std::error_code) { return -1; });
    
    // let_error example (returns a sender)
    auto retry_async = [](std::error_code) { return stdexec::just(7); };
    auto sndr = stdexec::just_error(std::error_code{ENOENT, std::system_category()})
                  | stdexec::let_error(retry_async);
  8. Configure GPU support with nvexec

    main

    For GPU offload using nvc++ -stdpar=gpu, use the schedulers provided in the <nvexec/...> headers:

    • nvexec::stream_scheduler (from <nvexec/stream_context.cuh>): A single-GPU scheduler for device 0.
    • nvexec::multi_gpu_stream_scheduler (from <nvexec/multi_gpu_context.cuh>): A multi-GPU scheduler that works across all visible devices.
  9. How sender consumers work

    main

    Consumers are the mechanism that actually executes a pipeline. A sender is merely a description of work; it does nothing until it is connected to a consumer.

    When choosing a consumer, consider two axes:

    1. Does the caller need to wait for the result?
    2. Who owns the lifetime of the operation state?
    ConsumerReturnsUse whenEager/Lazy
    stdexec::sync_waitstd::optional<std::tuple<...>>Top-level synchronous wait; single value-completion shape.lazy
    stdexec::sync_wait_with_variantstd::optional<std::variant<...>>Same, but sender has multiple value-completion shapes.lazy
    exec::start_detachedvoidTop-level fire-and-forget; no owning scope. (stdexec extension)eager
    stdexec::spawnvoidFire-and-forget into an async scope that will be joined later.eager
    stdexec::spawn_futuresenderSpawn into a scope and observe the result without blocking.eager

    Lifetime Ownership:

    • sync_wait / sync_wait_with_variant: The caller's stack frame owns the state.
    • start_detached: The operation owns itself (heap-allocated).
    • spawn / spawn_future: The async_scope owns the operation.
  10. How to choose between `starts_on`, `continues_on`, and `on`

    main

    When managing scheduler transitions in a pipeline, use these rules of thumb:

    • stdexec::starts_on: Use when starting a fresh pipeline and you want to stay on a specific scheduler.
    • stdexec::continues_on: Use when you want to hand off execution permanently to a new scheduler.
    • stdexec::on: Use when you want a "side trip" (run work on a different scheduler and then return to the original one).
  11. How to implement a custom scheduler

    main

    A scheduler in stdexec is a value-typed handle to an execution context. To satisfy the stdexec::scheduler concept, you must implement three structural pieces in a bottom-up fashion:

    1. Operation State: A type that holds the receiver and implements start(). It must opt into the operation state concept using using operation_state_concept = stdexec::operation_state_tag;. The start() method must be noexcept and return void.
    2. Schedule-Sender: The type returned by the scheduler's schedule() method. It must implement connect(R rcvr) which returns the operation state. It must also define completion_signatures (e.g., stdexec::completion_signatures<stdexec::set_value_t()>).
    3. Scheduler: A handle type that provides a schedule() member function returning the schedule-sender. It must be equality-comparable, copy-constructible, and nothrow-move-constructible.

    An 'inline scheduler' is the simplest implementation where start() calls stdexec::set_value directly on the calling thread, completing synchronously.

    // 1. Operation state
    template <stdexec::receiver R>
    struct simple_inline_opstate {
      using operation_state_concept = stdexec::operation_state_tag;
      R rcvr_;
      explicit simple_inline_opstate(R rcvr) noexcept : rcvr_(std::move(rcvr)) {}
      simple_inline_opstate(simple_inline_opstate&&) = delete;
      void start() noexcept {
        stdexec::set_value(std::move(rcvr_));
      }
    };
    
    // 2. Schedule-sender
    struct simple_inline_schedule_sender {
      using sender_concept = stdexec::sender_tag;
      using completion_signatures = stdexec::completion_signatures<stdexec::set_value_t()>;
    
      template <stdexec::receiver R>
      auto connect(R rcvr) const noexcept {
        return simple_inline_opstate<R>{std::move(rcvr)};
      }
    };
    
    // 3. Scheduler
    struct simple_inline_scheduler {
      auto schedule() const noexcept {
        return simple_inline_schedule_sender{};
      }
      bool operator==(simple_inline_scheduler const&) const noexcept = default;
    };
  12. Integrate senders with coroutines using `co_await`

    main

    Senders can be co_await-ed inside coroutines that use an awaitable-sender protocol (such as stdexec::task).

    Behavior:

    • Success: If the sender completes with a single successful completion shape, the coroutine resumes with the value.
    • Error: If the sender completes with an error, the coroutine throws an exception.
    • Stop: If the sender completes with a stop, the coroutine is canceled (it and its callers are destroyed and never resumed).

    All awaitable types can also be used as senders, allowing for seamless composition with sender algorithms.

    auto my_task() -> stdexec::task<int> {
          int x = co_await some_sender();
          co_return x + 1;
        }