Supabase Realtime Documentation

repository·main·Indexed 27 days ago

https://github.com/supabase/realtime

A high-performance server built with Elixir and Phoenix for real-time features including ephemeral messaging (Broadcast), shared state synchronization (Presence), and database change streaming (Postgres Changes) via WebSockets. Includes documentation for the Forum library, featuring Forum.Census for eventually-consistent distributed counting and Forum.Muster for group-routed fan-out broadcasts with efficient routing and rebalance lifecycle management.

Tokens
13.4K
Snippets
21
Records
55
Agent score
93%

What's inside Supabase Realtime

  1. Overview of Supabase Realtime features

    main

    Supabase Realtime is an Elixir-based server built with the Phoenix Framework that provides three core real-time capabilities over WebSockets:

    • Broadcast: Low-latency transmission of ephemeral messages from client to client.
    • Presence: Tracking and synchronizing shared state between connected clients.
    • Postgres Changes: Listening to changes in your Postgres database and streaming them to authorized clients.

    Note on Reliability: The server does not guarantee that every message will be delivered to all clients. It is designed for real-time updates rather than guaranteed message delivery protocols.

  2. Understand the Muster rebalance lifecycle and status

    main

    During a cluster membership change (nodes joining or leaving), a Forum.Muster scope transitions through a lifecycle managed by the coordinator. The cluster view is tracked via two persistent_term keys:

    • {Forum.Muster, scope, :status}: A tri-state indicating the current lifecycle phase:
      • :rebalancing: The ring is in flux; Muster.router/2 returns the full member list to ensure fan-out to everyone.
      • :converging: The ring has been adopted, but peers have not yet agreed on the new view.
      • :ready: Every peer agrees on the view; the occupancy table is considered trusted for routing.
    • {Forum.Muster, scope, :view_hash}: A hash (phash2) of the sorted member list used to detect view changes.

    Note that :ready implies the status is not :rebalancing.

  3. Understand Realtime metric scopes

    main

    Metrics are categorized by scope to help determine their granularity and intended use:

    • Per-Tenant: Metrics tagged with a tenant label. Used for individual tenant activity. Exposed on /tenant-metrics.
    • Global Aggregate: Metrics prefixed with realtime_channel_global_* or realtime_connections_global_*. These aggregate tenant data without a tenant label for cluster-wide dashboards. Exposed on /metrics.
    • Per-Node: Metrics measuring activity on the current Realtime node. If no per-node indication is present, assume they apply to the local node.
    • BEAM/Erlang VM: Metrics prefixed with beam_* and phoenix_* exposing Erlang runtime internals. Exposed on /metrics.
    • Infrastructure: Metrics prefixed with osmon_*, gen_rpc_*, and dist_* measuring system-level resources and cluster communication. Exposed on /metricsers.
  4. Forum.Muster for group-routed fan-out broadcast

    main

    Forum.Muster provides precise routing for fan-out broadcasts. Unlike simple broadcasting, Muster tracks which nodes hold local members for a specific group and designates a single router node for that group. This allows you to send a message to every process in a group across the cluster without blindly broadcasting to every node.

    Key Characteristics

    • Efficient Routing: Only routes to nodes that actually contain members of the group.
    • Ownership: Muster manages the routing decisions; the developer is responsible for providing the transport mechanism.
    • Partitioned: Membership is partitioned locally via Forum.Muster.Shard for concurrency.
  5. Use `Forum.Muster` for group-routed fan-out broadcast

    main

    Forum.Muster provides a mechanism to route broadcasts to specific nodes that contain members of a given group. Instead of broadcasting to every node in a cluster, it uses consistent hashing to identify a single router node for each group. The router maintains an authoritative set of nodes that must receive a broadcast for that group.

    Key Concepts

    • Router Node: Chosen via consistent hashing over the sorted member list. It owns the table of {group, node} mappings.
    • Coordinator: The per-node cluster brain managing membership, ring views, and rebalance orchestration.
    • Claim Shards: Processes that manage group membership and the state machine for claiming/releasing groups to the router. Shards use :erlang.phash2(group, N) to distribute work.
    • Transport: All inter-node communication (discovery, occupancy notifications, etc.) uses a pluggable transport module implementing the Forum.Adapter behaviour. By default, it uses Forum.Adapter.ErlDist.
  6. Configure Muster logging in standalone forum app

    main

    If you are running inside the standalone forum/ application (e.g., via iex -S mix), the default configuration suppresses log backends. To see logs in your console, you must manually attach a console handler and set the level to :debug.

    :logger.add_handler(:console, :logger_std_h, %{config: %{type: :standard_io}})
    :logger.set_primary_config(:level, :debug)
  7. Use Snabbkaffe for trace-based testing

    main

    The Snabbkaffe library allows you to assert on the trace of events emitted by a system and block until specific events occur. Trace points (tp/2,3) are compiled to near-zero cost in production and only collect data when MIX_ENV=test.

    # 1. Add trace points to your code
    defmodule Forum.Muster do
      use Snabbkaffe
    
      def rebalance_path(scope, members) do
        # ... logic
        tp(:muster_rebalance_done, %{scope: scope, members: members})
      end
    end
    
    # 2. Use check_trace/2 in your tests
    use Snabbkaffe
    
    test "rebalance converges" do
      check_trace(
        fn ->
          add_node(:node2)
          block_until(%{:"$kind" => :muster_rebalance_done}, 1000)
        end,
        fn trace ->
          assert [%{members: members}] = of_kind(:muster_rebalance_done, trace)
          assert :node2 in members
        end
      )
    end
  8. Listen to Postgres changes

    main

    To enable Realtime for a specific table, you must first create the table and then add it to the supabase_realtime publication in Postgres.

    create table test (
      id serial primary key
    );
    
    alter publication supabase_realtime add table test;
  9. Handle cluster growth with two-phase view adoption

    main

    To prevent missed deliveries during cluster growth, Forum.Muster uses a two-phase process:

    1. PREPARE (begin_view_change): The coordinator announces the move to a grown view to all members of the old committed view via note_transition RPCs. Recipients invalidate their member_views for the moving node. The coordinator only proceeds once all old-view members have acknowledged.
    2. COMMIT (do_rebalance): The coordinator swaps the ring and begins routing joins under the new view.

    If a membership change brings the cluster back to the original committed view before the commit phase completes, the round is cancelled (cancel_view_change), and the view_seq is bumped to ensure agreement is re-established.

    Shrink operations (nodes leaving) do not require a prepare phase and commit immediately.

  10. Access Realtime metrics endpoints

    main

    Supabase Realtime exposes metrics in Prometheus format via two primary endpoints. Accessing these endpoints requires a Bearer JWT token in the Authorization header, signed with the METRICS_JWT_SECRET.

    EndpointPriorityRecommended Scrape IntervalContents
    GET /metricsHigh30sBEAM/VM, OS, Phoenix, distributed infra, and global aggregated tenant totals (no tenant label)
    GET /tenant-metricsLow60sPer-tenant labeled metrics (connection counts, channel events, replication, authorization)
    GET /metrics/:regionHigh30sSame as /metrics scoped to a specific region
    GET /tenant-metrics/:regionLow60sSame as /tenant-metrics scoped to a specific region
    scrape_configs:
      - job_name: realtime_global
        scrape_interval: 30s
        bearer_token: <METRICS_JWT_SECRET_TOKEN>
        static_configs:
          - targets: ["<host>:4000"]
        metrics_path: /metrics
    
      - job_name: realtime_tenant
        scrape_interval: 60s
        bearer_token: <METRICS_JWT_SECRET_TOKEN>
        static_configs:
          - targets: ["<host>:4000"]
        metrics_path: /tenant-metrics
  11. Configure and start `Forum.Muster`

    main

    Start Forum.Muster under a supervisor. You must provide a scope name. Note that you should use a different scope name from any Census running on the same node.

    Configuration options include:

    • partitions: The number of claim shards (defaults to the number of online schedulers).
    • vacancy_cooldown_ms: How long a group stays in the cooldown state after the last member leaves (default: 30_000 ms).
    • vacant_flush_interval_ms: The interval at which shards flush batched :vacant_batch notifications to routers (default: 5_000 ms).
    • message_module: The module implementing Forum.Adapter used for RPC (default: Forum.Adapter.ErlDist).
    children = [
      {Forum.Muster,
       [:topics, partitions: 8, vacancy_cooldown_ms: 30_000, vacant_flush_interval_ms: 5_000]}
    ]