Watermill Documentation

Watermill is a lightweight Go library designed for building message-driven and event-driven applications. It provides a unified API to work with various Pub/Sub systems such as Kafka, RabbitMQ, and PostgreSQL, hiding the complexity of individual client libraries and connection management.

Note that Watermill is a library, not a framework, making it easy to integrate into existing projects or remove without significant architectural overhaul.

The way messages are consumed depends on the underlying Subscriber implementation:

1. Single stream of messages: The subscriber waits for msg.Ack() before receiving the next message. This is the simplest model.
2. Multiple message streams: Supported by some subscribers (like Kafka). They can subscribe to multiple partitions, allowing the Router to run concurrent HandlerFuncs for each partition, enabling parallel processing of messages even if previous ones haven't been acked.

The cqrs component provides abstractions for implementing Command-Query Responsibility Segregation. Key building blocks include:

• Events: Immutable representations of things that have already happened.
• Event Bus: Used for publishing events.
• Event Processor: Used for consuming and handling events.
• Commands: Data structures representing requests to execute an operation.
• Command Bus: Used for publishing commands.
• Command Processor: Used for consuming and handling commands.
• Marshaler: Handles the conversion between Go structs and messages.

The GoChannel Pub/Sub implementation uses Golang goroutines and channels to facilitate message passing. It is suitable for in-process communication where persistence is not required.

Characteristics

• Consumer Groups: Not supported.
• Exactly Once Delivery: Supported.
• Guaranteed Order: Supported.
• Persistence: Not persistent (in-memory).

Bolt is a pure Go key/value store (based on bbolt) suitable for projects that do not require a full database server like Postgres or MySQL.

Best use cases:

• Simple applications without external dependencies.
• Applications already using Bolt that want to publish messages within the same transaction used for saving other data.

Capabilities:

• Persistent: Yes
• Consumer Groups: No
• Exactly Once Delivery: No
• Guaranteed Order: No

To begin using Watermill, follow these steps:

1. Quickstart: Follow the Quickstart guide to learn by coding.
2. Guided Learning: Follow the Getting Started guide.
3. Explore Examples: Review the built-in examples in the repository, starting with _examples/basic/1-your-first-app.
4. Full Documentation: Refer to the official documentation at https://watermill.io/.

To ensure consistency between database state and event publishing, use the SQL Subscriber to save domain events in the same transaction as your aggregate within a SQL database (e.g., MySQL).

An asynchronous process then listens for new records in the SQL table and publishes them to a message broker like Kafka. This pattern prevents inconsistent states where a database update succeeds but the event publication fails, or vice versa.

Key components:

• SQL Subscriber: Listens for new records on a specific SQL table.
• Schema Adapter: Uses a schema (like DefaultMySQLSchema) to define how events are stored and retrieved from the database.
• Publisher: An external publisher (e.g., Kafka Publisher) that receives the events picked up by the SQL Subscriber.

A standard CQRS implementation in Watermill involves:

1. Sending a Command: Simulating an action (e.g., BookRoom) by sending a command object.
2. Command Handler: A component that processes the command and typically produces an event.
3. Event Handler: A component that reacts to events (e.g., RoomBooked) to trigger new commands or update read models.
4. Read Model: An asynchronous view of the system state generated by an event handler.

The Realtime Feed example demonstrates two microservices communicating over a Kafka topic. A producer service generates thousands of blog posts and publishes them to a Kafka topic, while a consumer service subscribes to that topic and displays the posts to standard output. The consumer includes throttling middleware to make the feed readable.

Requirements

• Docker
• docker-compose

docker-compose up