socketry/async

repository·main·Indexed 25 days ago

https://github.com/socketry/async

A Ruby package providing tools for concurrent programming focused on fiber-based cooperative multitasking. It includes abstractions such as Async::Task, Async::Reactor, and Async::Scheduler, along with utilities for concurrency control like Async::Semaphore, Async::Barrier, and Async::Idler. The library provides mechanisms for managing unbounded concurrency, implementing back-pressure via Async::Queue, and handling timeouts with task.with_timeout.

Tokens
27.9K
Snippets
73
Records
150
Agent score
78%

What's inside socketry-async

  1. Overview of Async

    main
    Async is a composable asynchronous I/O framework for Ruby built on top of io-event. It provides scalable event-driven I/O capable of handling thousands of clients per process using lightweight fiber-based concurrency. This approach allows for non-blocking I/O without the need for complex callback patterns. It also supports multi-thread and multi-process containers for parallelism.
  2. Avoid shared mutable state to prevent race conditions

    main

    The most fundamental cause of thread safety issues is shared mutable state (e.g., class instance variables, module variables, or shared mutable objects). In environments with multiple execution contexts like fibers or threads, shared mutable state creates unpredictable execution paths.

    Best Practices:

    • Avoid sharing mutable state whenever possible.
    • Prefer isolation, immutability, and pure functions.
    • If shared state is necessary, use coordination primitives like Mutex or concurrent data structures, but be aware of deadlocks and contention.
    class CurrencyConverter
    	def initialize
    		@exchange_rates = {} # Issue: Shared mutable state
    	end
    	
    	def update_rate(currency, rate)
    		# Issue: Multiple threads can modify @exchange_rates concurrently
    		@exchange_rates[currency] = rate
    	end
    	
    	def convert(amount, from_currency, to_currency)
    		# Issue: If @exchange_rates is modified while this method runs, it can lead to incorrect conversions
    		rate = @exchange_rates[from_currency] / @exchange_rates[to_currency]
    		amount * rate
    	end
    end
  3. Compare Fibers and Threads in Ruby

    main

    Fibers and threads are both primitives for concurrent execution, but they differ in how they are scheduled:

    Fibers

    • Cooperative multitasking: Usually runs within a single thread.
    • No preemption: A fiber must explicitly yield control. If a fiber executes a long-running task without yielding, it can cause latency issues for other fibers.
    • Explicit yield points: Control is yielded during I/O, Fiber.yield, sleep, etc.
    • Lightweight: Context switching is fast because it happens in user-space.
    • Limited parallelism: Parallelism is limited unless blocking operations are offloaded to a worker pool.

    Threads

    • Preemptive multitasking: Managed by the operating system and the Ruby thread scheduler.
    • Preemption: The interpreter can interrupt a thread at almost any point.
    • Expensive: Context switching has higher overhead due to OS involvement and interpreter contention.
    • Limited parallelism: Subject to the Ruby GVL (Global VM Lock) unless using rb_nogvl capable operations.
  4. Understand the Reactor Lifecycle and Task Completion

    main

    The Async reactor's event loop typically continues running until all tasks have completed. Completion is determined by Async::Task#finished?, which returns true only when the current node and all its children have finished execution.

    However, tasks can be marked as transient to change this behavior. Transient tasks do not keep the reactor alive; if only transient tasks remain at the root of the reactor, the reactor will exit.

  5. Core principles of thread safety in Ruby

    main

    When writing concurrent Ruby code using fibers or threads, follow these fundamental principles to prevent data corruption and race conditions:

    • Prioritize Data Integrity: Preventing data corruption is the most critical goal.
    • Default to Isolation: Avoid sharing mutable state. Prefer using pure functions, immutable objects, and dependency injection.
    • Assume Concurrency: Write code assuming it will be executed concurrently by multiple fibers, threads, or processes.
    • Assume Context Switching: Assume code may context switch at any time. Context switches occur most frequently during:
      • I/O operations: Network calls, file I/O, database queries, etc.
      • Explicit points: Fiber.yield, sleep, waiting on child processes, DNS queries, and signal handling (interrupts).
    • Beware of C Extensions: Native extensions (C/Rust) can block the fiber scheduler entirely. If a native operation is blocking, consider offloading it to a thread pool to avoid stalling the event loop.
  6. Optimistic vs Pessimistic Scheduling

    main

    The Async::Scheduler implementation detail regarding task execution order involves two strategies:

    1. Optimistic Scheduling: A greedy approach where tasks are executed as soon as they are scheduled via direct transfer of control flow. In an optimistic model, nested tasks may execute before the parent continues.
    2. Pessimistic Scheduling: Tasks are placed into the event loop's ready list and only executed during the next iteration of the event loop.

    Warning: You should not design your code to rely on a specific execution order, as the exact scheduling strategy is an unspecified implementation detail.

    ```ruby
    Async do
    	puts "Hello "
    	
    	Async do
    		puts "World"
    	end
    	
    	puts "!"
    end

    Optimistic: "Hello World!"

    Pessimistic: "Hello !World"

  7. Hierarchy of concurrency safety models

    main

    When designing concurrent systems, choose a safety model based on the level of coordination required. The hierarchy from safest to most complex is:

    1. No shared state (Ideal): Isolate state to each thread, fiber, or request. No coordination or synchronization is needed.
    2. Immutable shared state (Very good): Share data that does not change after creation, such as constants or frozen objects.
    3. Synchronized mutable state (Only when unavoidable): Share mutable state using robust synchronization mechanisms.

    Synchronization Strategies

    If you must use synchronized mutable state, choose the appropriate level of granularity:

    • Lock-free structures (e.g., Concurrent::Map): Provides safe, concurrent access with high performance and minimal contention.
    • Fine-grained locks: Protects the smallest necessary scope of shared state. Avoid holding these locks while yielding or running untrusted code.
    • Coarse-grained locks: Protects large areas of code or many data structures at once. Use these sparingly as they significantly reduce concurrency.
  8. How Async, Reactors, and Fibers work together

    main

    The async library relies on several core abstractions to manage concurrency:

    • Async::Task: Captures sequential computations. It runs using a Fiber and yields control during blocking operations (like sleep, read, or write), allowing other fibers to execute.
    • Async::Reactor: A specific implementation of a scheduler that includes an event loop and a selector. It manages the execution of fibers.
    • Fiber: The unit of cooperative concurrency. Execution can be transferred from one fiber to another and back.
    • Scheduler: An interface that manages fiber execution by intercepting blocking operations and redirecting them to an event loop.
    • Event Loop: The part of the scheduler responsible for waiting for events and waking up fibers when they are ready.
    • Selector: The component that interacts with the operating system to wait for specific events (e.g., file descriptor readiness).
  9. Avoid class variables (@@variable) for shared state

    main

    Class variables (@@variable) and class attributes (class_attribute) lack isolation because they are shared across the entire inheritance hierarchy. This can cause "spooky action at a distance" where modifying a variable in a parent class unexpectedly affects all child classes.

    Better alternatives:

    • Inject configuration or state through method parameters or constructor arguments.
    • Avoid using them if possible.
    class GlobalConfig
    	@@settings = {} # Issue: Class variables are shared across inheritance
    	
    	def set(key, value)
    		@@settings[key] = value
    	end
    	
    	def get(key)
    		@@settings[key]
    	end
    end
    
    class UserConfig < GlobalConfig
    end
    
    GlobalConfig.new.set(:foo, 42)
    # Issue: UserConfig inherits from GlobalConfig, so it shares the same @@settings:
    UserConfig.new.get(:foo) # => 42
  10. Understand the Reactor Lifecycle and Transient Tasks

    main

    The Async reactor's event loop typically continues running until all tasks have completed. This is determined by Async::Task#finished?, which checks if a task and all its children have finished execution.

    Transient Tasks are a special type of task that do not keep the reactor alive. They are intended for implementation details—background processes that support your application but are not core concurrency processes. If only transient tasks remain at the root of the reactor, the reactor will exit.

    Key behaviors of transient tasks:

    1. Reactor Exit: They do not prevent the reactor from shutting down. When all non-transient tasks finish, transient tasks are cancelled with an Async::Cancel exception.
    2. Tree Hoisting: If a parent task finishes or is cancelled, any transient child tasks are moved up the hierarchy to become children of the parent's parent. This prevents transient tasks from accidentally keeping a sub-tree of tasks alive.
  11. Manage unbounded concurrency with `Barrier`

    main

    A Barrier manages an unbounded number of tasks. The top-level Barrier method includes built-in load management via an Async::Idler, which prevents system overload by scheduling tasks only when system load is below 80%.

    Key Behaviors:

    • Automatic Cleanup: The barrier automatically waits for all tasks to complete and stops any outstanding tasks when the block exits.
    • Order of Completion: You can process tasks in the order they finish by calling barrier.wait with a block.
    • Disabling Load Management: To create tasks as fast as possible without the idler, pass parent: nil to the Barrier call.
    # Standard usage with load management
    Barrier do |barrier|
    	items.each do |item|
    		barrier.async do
    			process(item)
    		end
    	end
    end
    
    # Processing in order of completion
    Barrier do |barrier|
    	items.each do |item|
    		barrier.async do
    			process(item)
    		end
    	end
    	
    	barrier.wait do |task|
    		result = task.wait
    		# Do something with result.
    		# break if you don't want to wait for more tasks
    	end
    end
    
    # Disabling load management
    Barrier(parent: nil) do |barrier|
    	items.each do |item|
    		barrier.async do
    			process(item)
    		end
    	end
    end
    Barrier do |barrier|
    	items.each do |item|
    		barrier.async do
    			process(item)
    		end
    	end
    end