DOCSEARCHbyAlgolia
Submit Docs

ElasticMQ

repository·master·Indexed 25 days ago

https://github.com/softwaremill/elasticmq

An in-memory message queue system providing an Amazon SQS-compatible REST interface. Designed for testing and local development, it can be run as a standalone service via Docker, embedded in Scala applications, or as an actor-based system. Features include an optional monitoring UI, queue persistence via SQL databases, and support for FIFO queues.

Tokens
8.2K
Snippets
25
Records
52
Agent score
78%

What's inside ElasticMQ

  1. Overview of ElasticMQ

    master
    ElasticMQ is an in-memory message queue system that provides an Amazon SQS-compatible REST (query) interface. It is fully asynchronous and can be run as a standalone service via Docker, embedded in an application, or as a Scala actor-based system. It supports optional UI for monitoring and queue persistence.

  2. Build multi-architecture Docker images

    master
    You can build Docker images for both amd64 and arm64 using the Docker Buildx plugin. This requires creating a Buildx instance and using the --platform flag. Note that when using buildx, you must use the --push flag to store the image in a registry, as --load only supports single-architecture images.

  3. Run ElasticMQ locally with Docker Compose

    master

    The easiest way to run ElasticMQ locally is using Docker Compose. This starts both the SQS-compatible REST API and the Web UI.

    • SQS API: Port 9324 (softwaremill/elasticmq-native)
    • Web UI: Port 3000 (softwaremill/elasticmq-ui)

    After starting, access the UI at http://localhost:3000 to view queues, send messages, and monitor statistics. The configuration is loaded from examples/elasticmq.conf and data is persisted in a Docker volume named elasticmq-data.

    docker compose up

  4. Start an embedded ElasticMQ server in Scala

    master

    To use ElasticMQ as an embedded server within a Scala application, add the elasticmq-rest-sqs module to your build.sbt dependencies. You can start the server using a configuration file or via the SQSRestServerBuilder for more programmatic control.

    // Add to build.sbt
libraryDependencies += "org.elasticmq" %% "elasticmq-rest-sqs" % Version

// Option 1: Using a configuration file
val config = ConfigFactory.load("elasticmq.conf")
val server = new ElasticMQServer(new ElasticMQServerConfig(config))
server.start()

// Option 2: Using SQSRestServerBuilder
val server = SQSRestServerBuilder.start()
// ... use ...
server.stopAndWait()

  5. Manage ElasticMQ via the Web UI

    master

    ElasticMQ includes a web UI for monitoring and managing queues.

    • Docker Compose: The UI starts automatically at http://localhost:3000.
    • Local Development: Run npm install and npm run dev inside the ui directory. The dev server expects ElasticMQ at http://localhost:9324 (configurable via ui/.env.local).
    • Real AWS Integration: You can point the UI at a real AWS account by setting AWS_REGION, AWS_ACCESS_KEY_ID, and AWS_SECRET_ACCESS_KEY in ui/.env.local and leaving SQS_ENDPOINT unset.

  6. Quick Start: Running ElasticMQ Integration Tests (Node.js)

    master

    To run the integration tests for ElasticMQ using the AWS SDK for JavaScript v3, you can choose between using a local server (fastest) or Docker containers (full integration).

    Prerequisites

    • Node.js 18+
    • Docker (for container mode)
    • ElasticMQ Docker image (e.g., softwaremill/elasticmq:latest)

    Option 1: Local Server Mode (Fastest)

    1. Start ElasticMQ locally:
      docker run -p 9324:9324 softwaremill/elasticmq:latest
    2. Run tests:
      npm install
npm run test:local

    Option 2: Docker Container Mode (Full Integration)

    1. Pull the required image:
      docker pull softwaremill/elasticmq:latest
    2. Run tests:
      npm install
npm test
    # Local Server Mode
docker run -p 9324:9324 softwaremill/elasticmq:latest
npm install
npm run test:local

# Docker Container Mode
docker pull softwaremill/elasticmq:latest
npm install
npm test

  7. Add ElasticMQ dependencies (SBT and Maven)

    master

    To use the SQS-compatible REST interface, include the elasticmq-rest-sqs dependency. If you only need the actor-based core, use elasticmq-core.

    // Scala 2.13 and 2.12
val elasticmqSqs = "org.elasticmq" %% "elasticmq-rest-sqs" % Version

// Core only
val elasticmqCore = "org.elasticmq" %% "elasticmq-core" % Version
    <!-- Maven -->
<dependency>
    <groupId>org.elasticmq</groupId>
    <artifactId>elasticmq-rest-sqs_2.12</artifactId>
    <version>${version}</version>
</dependency>

  9. Automatically create queues on startup

    master

    You can pre-define queues and their attributes in a configuration file. Attributes like defaultVisibilityTimeout, delay, and deadLettersQueue are optional.

    To create a FIFO queue, set fifo = true. If the queue name ends with .fifo, the suffix is added automatically if not already present.

    Special attributes for testing:

    • copyTo: Duplicates messages to another queue.
    • moveTo: Redirects messages to another queue.
    queues {
  queue1 {
    defaultVisibilityTimeout = 10 seconds
    delay = 5 seconds
    receiveMessageWait = 0 seconds
    deadLettersQueue {
      name = "queue1-dead-letters"
      maxReceiveCount = 3
    }
    fifo = false
    contentBasedDeduplication = false
    copyTo = "audit-queue-name"
    moveTo = "redirect-queue-name"
    tags {
      tag1 = "tagged1"
      tag2 = "tagged2"
    }
  }
  queue1-dead-letters { }
  audit-queue-name { }
  redirect-queue-name { }
}

  12. Persist queue metadata to a configuration file

    master

    You can persist queue metadata (names and attributes, but not messages) to an external HOCON file. This file is updated whenever a queue is created, deleted, or modified. On startup, queues defined in this file take precedence over those in the main configuration.

    include classpath("application.conf")

queues-storage {
  enabled = true
  path = "/path/to/storage/queues.conf"
}