RedisSMQ Documentation

repository·next·Indexed 20 days ago

https://github.com/weyoss/redis-smq

A high-performance Redis-backed message queue for Node.js supporting FIFO, LIFO, and Priority queue types. Features include flexible routing, scheduling, rate limiting, and worker threads. The ecosystem includes the redis-smq-common library for foundational primitives like backoff strategies and worker management, a benchmarking suite for measuring producer and consumer throughput, and CI tools for changelog consolidation and GitHub release synchronization.

Tokens
202.8K
Snippets
653
Records
947
Agent score
69%

What's inside RedisSMQ

  1. Overview of RedisSMQ Common Library

    next
    RedisSMQ Common is a shared library providing core building blocks used across the entire RedisSMQ ecosystem. It contains essential utilities and shared components that support the main RedisSMQ functionality, including logging, Redis client management, server interactions, and backoff strategies for resilient operations.
  2. Overview of RedisSMQ

    next

    RedisSMQ is a high-performance, Redis-backed message queue library for Node.js. It provides a process-wide API for managing message flows using various queue types and routing models.

    Key Capabilities:

    • Queue Types: Supports FIFO, LIFO, and Priority queues.
    • Routing via Exchanges: Supports Direct, Topic, and Fanout exchanges.
    • Delivery Models: Supports both Point-to-Point and Pub/Sub models.
    • Advanced Features: Includes built-in scheduling (Delay, CRON, repeating), rate limiting, message auditing, and worker thread support for heavy handlers.
    • Reliability: Ensures atomic operations and at-least-once delivery guarantees.
  3. Overview of RedisSMQ REST API

    next

    The RedisSMQ REST API provides an HTTP interface that allows any web-capable application to interact with RedisSMQ message queues using a RESTful API.

    Key features include:

    • Strict request/response validation using JSON Schema.
    • Native OpenAPI v3 support and Swagger UI for API exploration.
    • Support for both ESM and CJS modules.
    • High test coverage (90%+).
  4. Overview of RedisSMQ

    next

    RedisSMQ is a Redis-backed Message Queue (MQ) designed for robustness and simplicity. It provides various queue and delivery models, along with producer/consumer APIs. It is designed to be production-ready and easy to operate, with optional integration for a REST API and Web UI to assist with monitoring and administration.

    Key features include:

    • Support for multiple Redis clients: ioredis or the official @redis/client.
    • Optional management tools: REST API and Web UI for observability and administration.
  5. Overview of RedisSMQ Web UI features

    next

    The RedisSMQ Web UI provides a graphical interface for managing your message queues. Key capabilities include:

    • Dashboard: View high-level stats for queues, consumers, and messages.
    • Browsing: Inspect queues and messages using filters.
    • Message Actions: Perform operations like ack (acknowledge), retry, and delete directly from the UI.
    • Model Support: Supports multiple queue and delivery models, including Direct, Topic, and Fanout exchange types.
    • Type Safety: Includes a type-safe OpenAPI client generated from the REST API schema.
  6. Features of RedisSMQ Web UI

    next

    The RedisSMQ Web UI provides the following capabilities:

    • Dashboard: View queues, consumers, and message statistics.
    • Browsers: Inspect queues and messages with support for filters and actions (e.g., ack, retry, delete).
    • Model Support: Supports multiple queue and delivery models, including Direct, Topic, and Fanout exchange types.
    • Type Safety: Includes a type-safe OpenAPI client generated from the REST API schema.
  7. Key features and capabilities of RedisSMQ

    next

    RedisSMQ is a high-performance, Redis-backed message queue for Node.js. It provides a simplified, process-wide API designed for low latency and operational simplicity.

    Core Capabilities:

    • Routing & Delivery: Supports Direct, Topic, and Fanout exchanges. Delivery models include Point-to-Point and Pub/Sub (using consumer groups).
    • Queue Strategies: Supports FIFO, LIFO, and Priority queues.
    • Scheduling & Throttling: Per-message scheduling (delay, CRON, repeat) and queue-level rate limiting.
    • Advanced Reliability: At-least-once delivery with acknowledgements, retries, and dead-lettering. Optional message audit for storing acknowledged and dead-lettered messages with retention policies.
    • Performance Optimizations: Uses Redis primitives and Lua scripts. Offers a direct queue publishing path (bypassing exchanges) for minimal overhead.
    • Execution Models: Optional worker thread execution to isolate message handlers from the main thread.
    • Observability: Optional EventBus for internal lifecycle and flow events.
    • Compatibility: Full support for both ESM and CJS module systems.
  8. What is an ExchangeFanout and when to use it

    next

    An ExchangeFanout is a fanout exchange used for broadcasting messages to all bound queues. It routes messages to every queue bound to the exchange while ignoring any routing keys. This is the ideal mechanism for implementing pub/sub (publisher/subscriber) patterns where every consumer is intended to receive the same message.

    const fanoutExchange = new ExchangeFanout();
    
    // Bind a queue
    await fanoutExchange.bindQueue('notifications', 'broadcast');
    
    // Match all bound queues
    const queues = await fanoutExchange.matchQueues('broadcast');
  9. Use Direct, Topic, or Fanout exchanges

    next

    RedisSMQ provides three exchange types for different routing requirements:

    1. Direct Exchange

    Routes messages to queues that have an exact match for the provided routing key.

    2. Topic Exchange

    Routes messages using pattern matching on dot-separated words:

    • * matches exactly one word.
    • # matches zero or more words.
    • Example: user.# matches user.login, user.login.success, etc.

    3. Fanout Exchange

    Broadcasts every message to every queue bound to the exchange. No routing key is required or used.

    // Direct
    const msg = new ProducibleMessage()
      .setDirectExchange('payments')
      .setExchangeRoutingKey('payment.processed')
      .setBody({ amount: 99.99 });
    
    // Topic
    const msg = new ProducibleMessage()
      .setTopicExchange('events')
      .setExchangeRoutingKey('user.login.success')
      .setBody({ userId: 123 });
    
    // Fanout
    const msg = new ProducibleMessage()
      .setFanoutExchange('alerts')
      .setBody({ alert: 'System down!' });
  10. Understand QueueOperationValidator and allowed operations

    next

    The QueueOperationValidator class is used to determine if specific operations are permitted on a queue based on its current state. Understanding these states is critical to preventing runtime errors when attempting to produce, consume, or manage queues.

    Queue StateAllowed Operations
    ACTIVEAll operations are permitted.
    PAUSEDAll operations are permitted except CONSUME.
    STOPPEDOnly management operations are permitted (e.g., purge, delete, rate limits, consumer groups, exchanges).
    LOCKEDNo operations are allowed.
  11. Configure message destinations (Queues and Exchanges)

    next

    A ProducibleMessage must have exactly one target destination. You can route messages directly to a queue or through various exchange types:

    | Target Type | Method | Routing Key Required? | | :--- | :--- | : | | Queue | .setQueue('name') | No | | Direct Exchange | .setDirectExchange('name') | Yes (via .setExchangeRoutingKey()) | | Topic Exchange | .setTopicExchange('name') | Yes (via .setExchangeRoutingKey()) | | Fanout Exchange | .setFanoutExchange('name') | No (Routing key is ignored) |

    // Direct Exchange example
    const msg = new ProducibleMessage()
      .setDirectExchange('orders')
      .setExchangeRoutingKey('order.created')
      .setBody({ orderId: '123' });
    
    // Topic Exchange example
    const msg = new ProducibleMessage()
      .setTopicExchange('events')
      .setExchangeRoutingKey('user.created')
      .setBody({ userId: 456 });
    
    // Fanout Exchange example
    const msg = new ProducibleMessage()
      .setFanoutExchange('notifications')
      .setBody({ alert: 'System update' });