DOCSEARCHbyAlgolia
Submit Docs

The Async Book

repository·master·Indexed 24 days ago

https://github.com/rust-lang/async-book

A comprehensive guide to asynchronous programming in Rust, covering the async/await syntax, the Future trait, and the async ecosystem. It explains core concepts such as inert futures, zero-cost abstractions, pinning, and the role of executors and runtimes like Tokio and async-std. The documentation provides comparisons between async and OS threads, guidance on avoiding common failure modes like blocking calls, and technical details on Wakers and Context.

Tokens
25.9K
Snippets
32
Records
142
Agent score
80%

What's inside async-book

  1. Overview of the Async Rust learning path

    master

    This guide provides a structured tutorial for learning asynchronous programming in Rust. The learning path progresses from fundamental concurrency models to advanced topics like synchronization, specialized tools, and async iterators (streams).

    Key learning stages include:

    1. Foundations: Understanding concurrency models (processes, threads, and async tasks) and the core async model.
    2. The Async/Await Paradigm: Learning the async/await syntax and programming patterns.
    3. I/O and Blocking: Mastering performant I/O and identifying/avoiding 'blocking' operations that prevent async progress.
    4. Concurrency Primitives & Synchronization: Learning how to abstract, compose, and synchronize concurrent code.
    5. Core Building Blocks: Deep dives into Futures and Runtimes.
    6. Advanced Topics: Async destruction/clean-up, timers, signal handling, and Streams (async iterators for sequences of events).

  2. What is structured concurrency?

    master

    Structured concurrency is a design philosophy for concurrent programs where tasks are organized into a hierarchical tree structure. In this model, every task (except the root) has exactly one parent, and child tasks must always finish executing before their parent task completes.

    Key Characteristics

    • Tree Organization: Tasks form a tree where each task has a single parent and no cycles exist.
    • Temporal Scope: A child task's lifetime is tied to its parent. This often follows lexical scope (the block where the task was created), but can be extended using objects like a 'nursery' or 'scope'.
    • Automatic Propagation: Because of the parent-child relationship, results and errors (including panics in Rust) are naturally passed back to the parent.
    • Mandatory Cancellation: If a parent task is cancelled, all its child tasks must also be cancelled, and their cancellation must complete before the parent finishes its own cancellation.
    • Predictable Resource Management: Since task lifetimes are bounded, it is easier to reason about when to clean up resources (like closing file handles) without relying solely on complex reference counting.

  3. What is a Future in Rust?

    master

    A future is the basic unit of async concurrency in Rust. It is a regular Rust object (a struct or enum) that implements the Future trait. A future represents a deferred computation—a value that will be available at some point in the future.

    Futures can be combined to create larger, more complex futures. While a 'task' is often used informally to describe a sequence of execution, in technical terms, an async task in Rust is simply a future that is being executed by a runtime.

  4. Design async programs using structured concurrency principles

    master

    To implement structured concurrency in async Rust, organize your program using a tree structure of parent and child tasks. Follow these design principles:

    • Temporal Scope: A function should not return (including via early returns or panics) until all tasks it launched are complete. Temporal scope should follow lexical scope where possible.
    • Data Flow: Primarily, data should flow from child tasks to parent tasks. Parent tasks should be responsible for handling the results and errors of their children.
    • Encapsulation: If writing a library, ensure temporal encapsulation. Do not start tasks that continue running after the API functions have returned.
    • Hybrid Approach: A common compromise is to allow unstructured concurrency only at the highest level (e.g., spawning tasks from main), while rigorously applying structured concurrency within each of those top-level tasks.

  5. How polling and cancellation work in async Rust

    master

    The execution of a future is driven by a process called Polling.

    • Polling: An executor calls the poll method on a future to check if it has completed. The result of a poll is a Poll type, which can be Poll::Ready(val) (the final state) or Poll::Pending (the future is still working).
    • Connection to await: The await syntax is the high-level way to interact with polling. When you await a future, you are essentially yielding control until the future returns Poll::Ready.
    • Cancellation via drop: In Rust, dropping a future is equivalent to cancelling it. If a future is dropped before it reaches the Ready state, its execution stops. This has significant implications for resource management and requires developers to be aware of cancellation safety to avoid leaving the system in an inconsistent state.

  6. How Task Wakeups work with `Waker`

    master

    In Rust's asynchronous model, futures often cannot complete on their first poll. To ensure a future is polled again once it is ready to make progress, the Waker type is used.

    Key concepts:

    • Tasks: The top-level futures submitted to an executor. Every time a future is polled, it is part of a task.
    • Waker::wake(): A method used to signal to the executor that the task associated with that Waker is ready to make progress and should be polled again.
    • Waker::clone(): Waker implements Clone, allowing it to be copied and stored (e.g., passed to a background thread or an I/O driver) so it can be invoked later to trigger a wakeup.

  7. Understand the `Stream` trait

    master
    The Stream trait is an asynchronous version of the standard library's Iterator trait. While a Future represents a single value that will eventually be available, a Stream can yield multiple values over time before finally completing. This makes it suitable for handling sequences of asynchronous events or data chunks.

  8. Understand the characteristics of Async in Rust

    master

    Rust's implementation of asynchronous programming has several unique characteristics that distinguish it from other languages:

    • Inert Futures: Futures in Rust do nothing unless they are polled. Dropping a future immediately stops its progress.
    • Zero-cost Abstractions: You can use async without mandatory heap allocations or dynamic dispatch, making it suitable for performance-critical or constrained environments like embedded systems.
    • No Built-in Runtime: The Rust standard library does not provide an async runtime. You must use community-maintained crates (e.g., Tokio, async-std) to execute async code.
    • Runtime Flexibility: You can choose between single-threaded or multi-threaded runtimes depending on your specific workload requirements.

  9. Use the `futures` crate for async utilities

    master

    The futures crate provides essential traits and functions for writing async code, including:

    • Stream
    • Sink
    • AsyncRead
    • AsyncWrite
    • Various combinators

    Note: The futures crate is not a full runtime because it includes an executor but lacks a reactor. It cannot support the execution of async I/O or timer futures on its own. It is common practice to use futures utilities in conjunction with an executor from a different crate.

  10. Prepare futures for `select!` using `Unpin` and `FusedFuture`

    master

    To use a future within a select! macro, it must satisfy two requirements:

    1. Unpin: select! takes futures by mutable reference rather than by value. This allows uncompleted futures to be reused in subsequent iterations of a loop. You can use pin_mut! to pin a future to the stack to satisfy this.
    2. FusedFuture: select! must not poll a future after it has already completed. The FusedFuture trait allows the macro to track completion status so it only polls futures that are still active. You can ensure a future implements this by calling .fuse() on it.

    For streams, there is a corresponding FusedStream trait. Streams that implement this or are wrapped with .fuse() will yield FusedFuture futures when using .next() or .try_next() combinators.

  11. Understand the core concept of Pinning

    master

    In Rust, Pin is a mechanism that marks a pointer as pointing to an object that is guaranteed not to move in memory until it is dropped.

    Why Pinning is necessary

    Pinning is essential for implementing async functions. Async functions are transformed into data structures (Futures) where variables are stored as fields. Because these variables may contain references to each other (self-references), the memory address of the object must remain stable. If the object moved, those internal references would point to invalid memory.

    Key takeaways for most developers

    • Safety: Pinning works by restricting access to mutable references to the pointee, preventing moves.
    • When you'll encounter it: You may need to interact with pinning when:
      • Manually calling Future::poll (which requires self to be pinned).
      • Using the select! macro, which may require pinning a reference using the pin! macro.
      • Implementing your own Future trait.

  12. How a Future executor works

    master

    In Rust, Futures are lazy and require an executor to drive them to completion. An executor manages a set of top-level Futures by calling their poll method.

    The Execution Lifecycle:

    1. Initial Poll: The executor typically calls poll once to start the future.
    2. Waiting: If the future cannot make progress, it returns Poll::Pending.
    3. Wakeup: When the future is ready to make progress again, it calls wake() on its Waker.
    4. Re-queueing: The Waker signals the executor (often by placing the task back onto a queue), and the executor calls poll again.
    5. Completion: This repeats until the future returns Poll::Ready.