EventBus

repository·main·Indexed 20 days ago

https://github.com/otobus/event_bus

A high-performance, minimalist event bus for Elixir that utilizes ETS for fast, concurrent event storage and watching. It features memory-efficient event shadows, support for static and dynamic topic registration, and observability compatible with the OpenTracing platform. The library includes the EventBus.EventSource module for a block-based builder pattern to simplify event creation and notification.

Tokens
4.8K
Snippets
20
Records
25
Agent score
68%

What's inside event_bus

  1. Overview of EventBus features

    main

    EventBus is a minimalist, traceable, and extendable event bus implementation for Elixir. It uses ETS for high-performance event storage and watching.

    Key characteristics:

    • Performance: Designed for fast concurrent reads and writes to ETS with near O(1) complexity for data access.
    • Memory Efficient: Instead of pushing full event data to all subscribers, it pushes an "event shadow" (containing only the event ID and topic) to interested subscribers.
    • Reliability: Applies queueing theory to handle inputs effectively.
    • Observability: Supports traceability with optional attributes compatible with the OpenTracing platform.
  2. Understand EventBus storage and ETS tables

    main

    EventBus uses ETS (Erlang Term Storage) tables for temporary event storage and status tracking. When an event is configured, two tables are created:

    1. Event Store (:eb_es_<<topic>>): A read-heavy table where event data is temporarily saved. Subscribers query this table to fetch the data they need to process.
    2. Event Watcher (:eb_ew_<<topic>>): A table used by the Observation Manager to track the status of event subscribers (who has processed, skipped, or is still pending).

    Important: ETS storage is not persistent. Data in both tables is automatically deleted by the Observation Manager once all subscribers have processed the event. To monitor unprocessed events, you can query the watcher table directly.

    To get a list of unprocessed events for a topic (e.g., :hello_received):

    # Returns a list of {id, {subscribers, completers, skippers}}
    :ets.tab2list(:eb_ew_hello_received)
  3. Install the EventBus library

    main

    To use event_bus in your Elixir project, add it to your deps in mix.exs and ensure it is included in your application's list of applications.

    # In mix.exs
    def deps do
      [
        {:event_bus, "~> 1.7.0"}
      ]
    end
    
    def application do
      [
        applications: [
          # ...
          :event_bus
        ]
      ]
    end
  4. Improve event traceability with optional fields

    main

    For highly traceable systems, it is recommended to populate optional fields within your event data.

    To automate the tracking of event timing, use the EventSource.notify block or yield. This will automatically set the following values:

    • initialized_at
    • occurred_at
  5. Build and notify events using EventSource

    main

    The EventBus.EventSource module provides a block-based builder pattern to simplify event creation and notification. It automatically handles timestamps (initialized_at, occurred_at) and can handle errors by routing them to an error_topic.

    Using EventSource.build/2

    Use this to create an %Event{} struct from a block of logic. The return value of the block becomes the event's data.

    Using EventSource.notify/2

    Use this to execute logic and immediately notify subscribers with the result. If the block returns an {:error, reason} tuple, the event is automatically published to the error_topic (if provided) instead of the primary topic.

    use EventBus.EventSource
    
    # 1. Build an event struct
    params = %{id: "unique_id", topic: :user_created, source: "my_app"}
    EventSource.build(params) do
      # Logic here
      %{email: "user@example.com"}
    end
    
    # 2. Notify subscribers directly
    params = %{id: "unique_id", topic: :user_created, error_topic: :user_error}
    EventSource.notify(params) do
      # If this returns {:error, reason}, it goes to :user_error
      {:error, %{reason: "failed"}}
    end
  6. Implement persistent event storage using a wildcard subscriber

    main

    Because EventBus uses ETS for temporary storage, you must implement your own persistence logic if you need to keep event data long-term.

    The recommended pattern is to subscribe a dedicated module to all event types using the [."*" ] wildcard topic pattern, then save the fetched event data to a persistent database.

    Follow this pattern:

    1. Subscribe to [."*" ].
    2. In the process/1 callback, fetch the event data using EventBus.fetch_event/1.
    3. Save the data to your persistent store.
    4. Call EventBus.mark_as_completed/1 to notify the Observation Manager.
    # 1. Subscribe to all topics
    EventBus.subscribe({MyDataStore, [.".*" ]})
    
    # 2. Implement the subscriber module
    defmodule MyDataStore do
      # The process/1 callback receives the event shadow
      def process({topic, id} = event_shadow) do
        GenServer.cast(__MODULE__, event_shadow)
        :ok
      end
    
      def handle_cast({topic, id}, state) do
        # 3. Fetch the actual event data from ETS
        event = EventBus.fetch_event({topic, id})
        
        # 4. Write logic to save event_data to a persistent store (e.g. Postgres)
        # ...
    
        # 5. Mark as completed so the Observation Manager can clean up ETS
        EventBus.mark_as_completed({__MODULE__, {topic, id}})
        {:noreply, state}
      end
    end
    EventBus.subscribe({MyDataStore, [.".*" ]})
    
    defmodule MyDataStore do
      def process({topic, id} = event_shadow) do
        GenServer.cast(__MODULE__, event_shadow)
        :ok
      end
    
      def handle_cast({topic, id}, state) do
        event = EventBus.fetch_event({topic, id})
        # write your logic to save event_data to a persistent store
    
        EventBus.mark_as_completed({__MODULE__, {topic, id}})
        {:noreply, state}
      end
    end
  7. Configure EventBus defaults

    main

    You can configure global defaults for EventBus in your application configuration. This allows EventSource to automatically generate IDs, timestamps, and TTLs without manual input.

    config :event_bus,
      topics: [],             # list of atoms
      ttl: 30_000_000,        # integer
      time_unit: :microsecond, # atom
      id_generator: EventBus.Util.Base62 # module implementing unique_id/0
  8. List subscribers

    main

    You can inspect active subscribers using the following methods:

    • EventBus.subscribers(): Returns a list of all current subscribers and their topic patterns.
    • EventBus.subscribers(topic): Returns a list of subscribers specifically interested in the provided topic.
    # List all subscribers
    EventBus.subscribers()
    # Returns: [{MyEventSubscriber, [".*"]}, {{AnotherSubscriber, %{}}, [".*"]}]
    
    # List subscribers for a specific topic
    EventBus.subscribers(:hello_received)
    # Returns: [MyEventSubscriber, {{AnotherSubscriber, %{}}}]
  9. Unsubscribe from the event bus

    main

    To stop a subscriber from receiving events, use EventBus.unsubscribe/1. You can pass either the module name or the tuple containing the configuration used during subscription.

    # Unsubscribe by module name
    EventBus.unsubscribe(MyEventSubscriber)
    
    # Unsubscribe by module and config tuple
    config = %{}
    EventBus.unsubscribe({MyEventSubscriber, config})
  10. Register and unregister event topics

    main

    You can manage event topics in two ways:

    1. Static Registration: Define topics in your config.exs file.
    2. Dynamic Registration: Use EventBus.register_topic/1 and EventBus.unregister_topic/1 at runtime.

    Warning: Calling EventBus.unregister_topic/1 also deletes the related topic tables in the database.

    # Static registration in config.exs
    config :event_bus, topics: [:message_received, :another_event_occurred]
    
    # Dynamic registration
    EventBus.register_topic(:webhook_received)
    
    # Dynamic unregistration (Warning: deletes topic tables!)
    EventBus.unregister_topic(:webhook_received)
  11. Fetch events from the store

    main

    You can retrieve specific events or just their data from the event store using the topic and the unique event ID.

    topic = :bye_received
    id = "124"
    
    # Fetch the full event struct
    EventBus.fetch_event({topic, id})
    
    # Fetch only the event data payload
    EventBus.fetch_event_data({topic, id})
  12. Notify subscribers with an Event

    main

    To broadcast an event to all interested subscribers, use EventBus.notify/1 with an %EventBus.Model.Event{} struct.

    alias EventBus.Model.Event
    
    event = %Event{id: "123", topic: :hello_received, data: %{message: "Hello"}}
    EventBus.notify(event)