redis-semaphore

repository·master·Indexed 42 days ago

https://github.com/dv/redis-semaphore

A Ruby implementation of mutexes and semaphores using Redis. It utilizes the blocking BLPOP command to provide a fair, first-come-first-serve queue for processes waiting on resources. Features include support for blocking and non-blocking locks, stale lock handling via stale_client_timeout, and distributed resource management.

Tokens
2.7K
Snippets
12
Records
12
Agent score
54%

What's inside redis-semaphore

  1. Handle stale locks with stale_client_timeout

    master

    To prevent dead clients from holding locks forever, you can use a stale-check mechanism. If a lock is older than the specified timeout, it can be considered stale.

    Option 1: Automatic check on lock

    Pass :stale_client_timeout (in seconds) during initialization. The semaphore will check for stale locks every time you attempt to call lock.

    Option 2: Watchdog thread

    Run a separate thread that calls release_stale_locks! frequently. This is more effective as it can unblock processes currently waiting in a lock call.

    # Option 1: Automatic check
    s = Redis::Semaphore.new(:stale_semaphore, :stale_client_timeout => 5)
    
    # Option 2: Watchdog thread
    Thread.new do
      watchdog = Redis::Semaphore.new(:semaphore, :stale_client_timeout => 5)
      while(true)
        watchdog.release_stale_locks!
        sleep 1
      end
    end
  2. How to lock and unlock resources

    master

    You can use lock with a block for automatic management, or call lock and unlock linearly. When using linear calls, you must call unlock to prevent the resource from staying locked indefinitely.

    Blocking Behavior (v0.3.0+)

    • lock: Blocks indefinitely until a resource is available.
    • lock(nil): Blocks indefinitely (legacy behavior).
    • lock(0): Non-blocking. Returns immediately if no resource is available.
    • lock(timeout): Blocks for a maximum of timeout seconds.
    # Using a block (recommended: auto-unlocks even on exceptions)
    s.lock do
      work
    end
    
    # Linear calls (manual unlock required)
    s.lock
    work
    s.unlock
    
    # Non-blocking attempt
    if s.lock(0)
      work
      s.unlock
    else
      puts "No resource available"
    end
    
    # Blocking with a timeout
    if s.lock(5)
      work
      s.unlock
    else
      puts "Timed out"
    end
  3. Configure expiration and local time

    master

    Expiration

    Pass :expiration (in seconds) to the constructor. This uses the Redis EXPIRE command on semaphore keys to ensure they are eventually cleaned up. Warning: Use this carefully; if the expiration is too short, the semaphore might disappear while a client is still in a critical section.

    Local Time

    By default, the library uses the Redis server's time (via the TIME command) to ensure consistency across distributed clients. To save a network round-trip and use the client's local clock instead, set :use_local_time => true.

    # With expiration and local time
    s = Redis::Semaphore.new(:name, :expiration => 100, :use_local_time => true)
  4. Use wait and signal for advanced semaphore patterns

    master

    The wait and signal methods provide traditional semaphore functionality.

    • wait: Retrieves a resource (aliased to lock).
    • signal: Returns a resource to the semaphore. If you pass a specific token, that token is returned. If no token is passed, a new unique token is generated.

    This is useful for producer/consumer patterns where one process acquires a resource and another releases it.

    # Consumer process
    token = sem.wait
    
    # Producer process
    sem.signal(token) # Returns the specific token
    # OR
    sem.signal         # Returns a new unique token
  5. Create a Mutex or Semaphore

    master

    Initialize a new Redis::Semaphore instance. You can provide a name, a host, or a custom Redis client.

    To create a Mutex (only one process allowed), omit the :resources option. To create a Semaphore (multiple processes allowed), set the :resources option to the desired number of available slots.

    Note: It is recommended to use a unique Redis client per thread because the underlying BLPOP command is blocking.

    # Create a Mutex
    s = Redis::Semaphore.new(:semaphore_name, :host => "localhost")
    
    # Create a Semaphore with 5 resources
    s = Redis::Semaphore.new(:semaphore_name, :resources => 5, :host => "localhost")
    
    # Use a custom Redis client
    r = Redis.new(:host => "localhost", :db => 222)
    s = Redis::Semaphore.new(:another_name, :redis => r)
  6. Check semaphore status with exists?, locked?, and available_count

    master

    Query the state of the semaphore using these methods:

    • exists?: Returns true if the semaphore has been initialized in Redis.
    • locked?(token = nil): Returns true if the semaphore is currently held. If a token is provided, it checks if that specific token is currently held. If no token is provided, it checks if any tokens are held.
    • available_count: Returns the number of resources currently available for acquisition.
    if semaphore.exists?
      puts "Available resources: #{semaphore.available_count}"
    end
    
    if semaphore.locked?
      puts "Semaphore is currently in use"
    end
  7. Initialize Redis::Semaphore

    master

    Create a new semaphore instance by providing a unique name and an optional configuration hash. You can specify the number of available resources, a stale client timeout, an expiration time for keys, or a pre-existing Redis connection.

    Key options:

    • :resources (Integer): The number of concurrent resources available. Defaults to 1.
    • :stale_client_timeout (Numeric): The threshold of time (in seconds) before a client is assumed to be dead and its lock is invalidated. If nil (default), stale clients are not checked.
    • :expiration (Numeric): Sets an expiration time on the semaphore's Redis keys.
    • :redis (Redis instance): An existing Redis connection. If not provided, a new Redis instance is created using the remaining options (e.g., :host, :port, :path).
    • :use_local_time (Boolean): If true, uses the local system time instead of Redis server time for synchronization.
    # Basic usage with default settings
    semaphore = Redis::Semaphore.new(:my_semaphore, :redis => my_redis_connection)
    
    # Usage with specific resource count and stale client detection
    semaphore = Redis::Semaphore.new(:my_semaphore, :resources => 5, :stale_client_timeout => 30, :redis => my_redis_connection)
    
    # Usage with connection details instead of a connection object
    semaphore = Redis::Semaphore.new(:my_semaphore, :host => "localhost", :port => 6379)
  8. Acquire and release locks with lock() and unlock()

    master

    Use lock to acquire a resource. You can pass a timeout to control how long the process should wait for a resource.

    • Blocking mode: If timeout is nil or > 0, the method blocks using Redis BLPOP until a resource is available or the timeout expires.
    • Non-blocking mode: If timeout is 0 or less, the method returns immediately if no resource is available.
    • Block syntax: If a block is provided to lock, the semaphore is automatically released (signaled) when the block completes, even if an error occurs.

    unlock manually releases a held token. If using the block syntax, manual unlocking is not required.

    wait is an alias for lock.

    # 1. Blocking lock with automatic release
    semaphore.lock do |token|
      puts "Acquired token: #{token}"
      # Perform critical section work
    end
    
    # 2. Non-blocking lock (returns immediately)
    token = semaphore.lock(0)
    if token
      begin
        # Do work
      ensure
        semaphore.unlock
      end
    else
      puts "No resource available"
    end
    
    # 3. Blocking lock with specific timeout (seconds)
    semaphore.lock(10) do |token|
      # Do work
    end
  9. Use Redis::Semaphore to manage distributed locks

    master

    The Redis::Semaphore class provides a way to implement distributed semaphores (and mutexes) using Redis. It allows you to limit the number of concurrent processes that can execute a specific block of code.

    To use it, instantiate a Redis::Semaphore with a Redis client and a unique name. You can then use #acquire to block until a slot is available, or #try_acquire to attempt to acquire a slot without blocking.

    # Example of basic usage
    semaphore = Redis::Semaphore.new(redis_client, name: 'my_resource_lock', limit: 5)
    
    # Blocking acquire
    semaphore.acquire do
      # Critical section: only up to 5 processes can be here at once
    end
    
    # Non-blocking attempt
    if semaphore.try_acquire
      begin
        # Critical section
      ensure
        semaphore.release
      end
    else
      puts "Resource busy"
    end