Concurrent Ruby

repository·master·Indexed 26 days ago

https://github.com/ruby-concurrency/concurrent-ruby

A comprehensive suite of modern concurrency tools and abstractions for Ruby, providing thread-safe implementations of patterns inspired by Erlang, Java, Clojure, and Scala. It supports MRI/CRuby, JRuby, and TruffleRuby, offering high-level abstractions like Promises, ScheduledTask, and TimerTask, thread-safe collections (Map, Array, Hash), and state management tools such as Atoms and Agents. Experimental features, including the Actor model and CSP-style Channels, are available via the concurrent-ruby-edge gem.

Tokens
23K
Snippets
54
Records
133
Agent score
88%

What's inside concurrent-ruby

  1. Overview of the Promises framework

    master

    The Promises framework in concurrent-ruby unifies several older concurrency tools (Concurrent::Future, Concurrent::Promise, Concurrent::IVar, Concurrent::Event, Concurrent.dataflow, Delay, and TimerTask) into a single, cohesive system.

    Key capabilities include:

    • Asynchronous Task Processing: Process tasks without blocking the main execution flow.
    • Task Composition: Chain, branch, and zip asynchronous tasks to create Directed Acyclic Graphs (DAGs).
    • Scheduling: Create delayed or scheduled tasks.
    • Error Handling: Manage failures through rejections.
    • Concurrency Control: Manage the level of concurrency and simulate massive amounts of parallel processing (tens of thousands of tasks) on a single thread pool without occupying individual threads.
    • Integration: Seamlessly combine promises with actors and channels.
    • Back Pressure: Build parallel processing stream systems that can signal components to slow down when they cannot keep up.
  2. Overview of Concurrent Ruby

    master
    Concurrent Ruby is a toolbox of modern concurrency utilities for Ruby, inspired by patterns from Erlang, Clojure, Scala, Haskell, F#, C#, and Java. It is designed to be an unopinionated, lean, and loosely coupled library that provides thread-safe abstractions while remaining free of external gem dependencies. It aims to provide idiomatic Ruby implementations of classic concurrency patterns.
  3. Use Actor models

    master

    The Actor abstraction is designed to match Erlang-like behavior. The implementation supports different modes depending on the use case:

    • Thread-backed: Each actor is backed by its own thread. Best for a limited number of long-running actors where a simple actor body is desired.
    • Stack-less: Each message triggers a method call on the actor. These can run on a thread pool, allowing for a large number of short-lived actors, but they are less suitable for complex bodies that change behavior.
    • Simulated process: Offers the most flexibility with no limitations, but requires more complex actor body implementations.
  4. Use General-purpose Concurrency Abstractions

    master

    Concurrent Ruby provides several high-level abstractions for managing asynchronous tasks:

    • Async: A mixin module for providing simple asynchronous behavior to a class (inspired by Erlang's gen_server).
    • ScheduledTask: A task scheduled to run at a specific future time.
    • TimerTask: A thread that periodically performs work at regular intervals.
    • Promises: A unified, non-blocking, and lock-free framework that replaces Future, Promise, IVar, Event, dataflow, and Delay. It is the recommended way to handle asynchronous values and dependencies.
  5. Chain tasks using .then and .chain

    master

    Promises allow non-blocking task composition:

    • .then { |val| ... }: Executes the block only if the preceding future is fulfilled. The result of the previous future is passed as an argument.
    • .chain { |fulfilled, value, reason| ... }: Executes the block regardless of whether the preceding future was fulfilled or rejected. The block receives the status, the value (if fulfilled), and the reason (if rejected).
    • .rescue { |err| ... }: Executes the block only if the preceding future is rejected. This is used to recover from errors and return a new fulfilled value.
  6. Run the Top Stock example

    master

    The Top Stock example demonstrates how to concurrently fetch stock data from the Alpha Vantage service to determine which stock had the highest closing price in a given year.

    To run this example, you must first obtain a free API key from Alpha Vantage. Then, execute the script from the root of the repository providing your key via the ALPHAVANTAGE_KEY environment variable.

    $ ALPHAVANTAGE_KEY=YOUR_API_KEY bundle exec ruby top-stock-scala/top-stock.rb
  7. Create a periodic task using schedule and Cancellation

    master

    You can implement a repeating task by combining Concurrent::Promises.schedule, run, and Concurrent::Cancellation. The pattern involves scheduling a task and then using .then to recursively call the scheduler, creating a loop that respects the cancellation token.

    To stop the periodic task, call .resolve on the origin object provided by Concurrent::Cancellation.new.

    repeating_scheduled_task = -> interval, cancellation, task do
      Concurrent::Promises.
          # Schedule the task.
          schedule(interval, cancellation, &task).
          # If successful, schedule again to create a loop.
          then { repeating_scheduled_task.call(interval, cancellation, task) }
    end
    
    cancellation, origin = Concurrent::Cancellation.new
    
    task = -> cancellation do
      5.times do
        cancellation.check! # Ensure we stop if canceled
        do_stuff
      end
    end
    
    # Start the periodic task
    result = Concurrent::Promises.future(0.1, cancellation, task, &repeating_scheduled_task).run
    
    # Stop the task
    origin.resolve
  8. Understand Concurrent Ruby versioning and stability

    master

    The project follows Semantic Versioning (SemVer) with specific rules for its components:

    • concurrent-ruby: Uses standard Semantic Versioning.
    • concurrent-ruby-ext: Always matches the version of concurrent-ruby.
    • concurrent-ruby-edge: Uses 0.y.z versioning. While it follows SemVer, minor version increments indicate incompatible changes, and patch increments indicate compatible changes. Because it is in initial development, the public API is not considered stable and may change at any time.
  9. Integrate Concurrent::Promises::FactoryMethods into classes or modules

    master

    The Concurrent::Promises::FactoryMethods module provides constructor methods for creating Future and Event objects. Instead of direct inheritance, use composition by including or extending this module.

    By default, Concurrent::Promises is already extended with these methods. You can also override the default_executor within a module that extends FactoryMethods to change the execution context for tasks created via that module.

    # Including in a class
    Class.new do
      include Concurrent::Promises::FactoryMethods
      def a_method
        resolvable_event
      end
    end.new.a_method
    
    # Extending a module and overriding the executor
    mod = Module.new do
      extend Concurrent::Promises::FactoryMethods
      def self.default_executor
        :fast
      end
    end 
    mod.future { 1 }.default_executor # => :fast
  10. Choose between :on_thread and :on_pool actor types

    master

    When spawning an actor, choose a type based on your scale requirements:

    • :on_thread: The receive method blocks the actor's thread until a message is available. This is simpler to write but each actor consumes a full Ruby thread. Use this for a small number of actors.
    • :on_pool: The receive method returns immediately, freeing the thread back to a pool. You must provide a block (or blocks) to receive that acts as a continuation to be executed when a message arrives. This is much more efficient for hundreds or thousands of short-lived actors as it is limited by RAM rather than thread count.