RedisSMQ Documentation
repository·next·Indexed 20 days ago
https://github.com/weyoss/redis-smqA high-performance Redis-backed message queue for Node.js supporting FIFO, LIFO, and Priority queue types. Features include flexible routing, scheduling, rate limiting, and worker threads. The ecosystem includes the redis-smq-common library for foundational primitives like backoff strategies and worker management, a benchmarking suite for measuring producer and consumer throughput, and CI tools for changelog consolidation and GitHub release synchronization.
What's inside RedisSMQ
- RedisSMQ Common is a shared library providing core building blocks used across the entire RedisSMQ ecosystem. It contains essential utilities and shared components that support the main RedisSMQ functionality, including logging, Redis client management, server interactions, and backoff strategies for resilient operations.
Overview of RedisSMQ
nextRedisSMQ is a high-performance, Redis-backed message queue library for Node.js. It provides a process-wide API for managing message flows using various queue types and routing models.
Key Capabilities:
- Queue Types: Supports FIFO, LIFO, and Priority queues.
- Routing via Exchanges: Supports Direct, Topic, and Fanout exchanges.
- Delivery Models: Supports both Point-to-Point and Pub/Sub models.
- Advanced Features: Includes built-in scheduling (Delay, CRON, repeating), rate limiting, message auditing, and worker thread support for heavy handlers.
- Reliability: Ensures atomic operations and at-least-once delivery guarantees.
Overview of RedisSMQ REST API
nextThe RedisSMQ REST API provides an HTTP interface that allows any web-capable application to interact with RedisSMQ message queues using a RESTful API.
Key features include:
- Strict request/response validation using JSON Schema.
- Native OpenAPI v3 support and Swagger UI for API exploration.
- Support for both ESM and CJS modules.
- High test coverage (90%+).
Overview of RedisSMQ
nextRedisSMQ is a Redis-backed Message Queue (MQ) designed for robustness and simplicity. It provides various queue and delivery models, along with producer/consumer APIs. It is designed to be production-ready and easy to operate, with optional integration for a REST API and Web UI to assist with monitoring and administration.
Key features include:
- Support for multiple Redis clients:
ioredisor the official@redis/client. - Optional management tools: REST API and Web UI for observability and administration.
- Support for multiple Redis clients:
Overview of RedisSMQ Web UI features
nextThe RedisSMQ Web UI provides a graphical interface for managing your message queues. Key capabilities include:
- Dashboard: View high-level stats for queues, consumers, and messages.
- Browsing: Inspect queues and messages using filters.
- Message Actions: Perform operations like
ack(acknowledge),retry, anddeletedirectly from the UI. - Model Support: Supports multiple queue and delivery models, including Direct, Topic, and Fanout exchange types.
- Type Safety: Includes a type-safe OpenAPI client generated from the REST API schema.
Explore the RedisSMQ API Reference
nextThe RedisSMQ API is organized into several categories including Namespaces, Enumerations, Classes, Interfaces, Type Aliases, and Variables. Use these categories to locate specific functionality for managing queues, producers, consumers, and exchanges.Features of RedisSMQ Web UI
nextThe RedisSMQ Web UI provides the following capabilities:
- Dashboard: View queues, consumers, and message statistics.
- Browsers: Inspect queues and messages with support for filters and actions (e.g.,
ack,retry,delete). - Model Support: Supports multiple queue and delivery models, including Direct, Topic, and Fanout exchange types.
- Type Safety: Includes a type-safe OpenAPI client generated from the REST API schema.
Key features and capabilities of RedisSMQ
nextRedisSMQ is a high-performance, Redis-backed message queue for Node.js. It provides a simplified, process-wide API designed for low latency and operational simplicity.
Core Capabilities:
- Routing & Delivery: Supports Direct, Topic, and Fanout exchanges. Delivery models include Point-to-Point and Pub/Sub (using consumer groups).
- Queue Strategies: Supports FIFO, LIFO, and Priority queues.
- Scheduling & Throttling: Per-message scheduling (delay, CRON, repeat) and queue-level rate limiting.
- Advanced Reliability: At-least-once delivery with acknowledgements, retries, and dead-lettering. Optional message audit for storing acknowledged and dead-lettered messages with retention policies.
- Performance Optimizations: Uses Redis primitives and Lua scripts. Offers a direct queue publishing path (bypassing exchanges) for minimal overhead.
- Execution Models: Optional worker thread execution to isolate message handlers from the main thread.
- Observability: Optional EventBus for internal lifecycle and flow events.
- Compatibility: Full support for both ESM and CJS module systems.
What is an ExchangeFanout and when to use it
nextAn
ExchangeFanoutis a fanout exchange used for broadcasting messages to all bound queues. It routes messages to every queue bound to the exchange while ignoring any routing keys. This is the ideal mechanism for implementing pub/sub (publisher/subscriber) patterns where every consumer is intended to receive the same message.const fanoutExchange = new ExchangeFanout(); // Bind a queue await fanoutExchange.bindQueue('notifications', 'broadcast'); // Match all bound queues const queues = await fanoutExchange.matchQueues('broadcast');Use Direct, Topic, or Fanout exchanges
nextRedisSMQ provides three exchange types for different routing requirements:
1. Direct Exchange
Routes messages to queues that have an exact match for the provided routing key.
2. Topic Exchange
Routes messages using pattern matching on dot-separated words:
*matches exactly one word.#matches zero or more words.- Example:
user.#matchesuser.login,user.login.success, etc.
3. Fanout Exchange
Broadcasts every message to every queue bound to the exchange. No routing key is required or used.
// Direct const msg = new ProducibleMessage() .setDirectExchange('payments') .setExchangeRoutingKey('payment.processed') .setBody({ amount: 99.99 }); // Topic const msg = new ProducibleMessage() .setTopicExchange('events') .setExchangeRoutingKey('user.login.success') .setBody({ userId: 123 }); // Fanout const msg = new ProducibleMessage() .setFanoutExchange('alerts') .setBody({ alert: 'System down!' });Understand QueueOperationValidator and allowed operations
nextThe
QueueOperationValidatorclass is used to determine if specific operations are permitted on a queue based on its current state. Understanding these states is critical to preventing runtime errors when attempting to produce, consume, or manage queues.Queue State Allowed Operations ACTIVE All operations are permitted. PAUSED All operations are permitted except CONSUME.STOPPED Only management operations are permitted (e.g., purge,delete,rate limits,consumer groups,exchanges).LOCKED No operations are allowed. Configure message destinations (Queues and Exchanges)
nextA
ProducibleMessagemust have exactly one target destination. You can route messages directly to a queue or through various exchange types:| Target Type | Method | Routing Key Required? | | :--- | :--- | : | | Queue |
.setQueue('name')| No | | Direct Exchange |.setDirectExchange('name')| Yes (via.setExchangeRoutingKey()) | | Topic Exchange |.setTopicExchange('name')| Yes (via.setExchangeRoutingKey()) | | Fanout Exchange |.setFanoutExchange('name')| No (Routing key is ignored) |// Direct Exchange example const msg = new ProducibleMessage() .setDirectExchange('orders') .setExchangeRoutingKey('order.created') .setBody({ orderId: '123' }); // Topic Exchange example const msg = new ProducibleMessage() .setTopicExchange('events') .setExchangeRoutingKey('user.created') .setBody({ userId: 456 }); // Fanout Exchange example const msg = new ProducibleMessage() .setFanoutExchange('notifications') .setBody({ alert: 'System update' });