Synapse Documentation

repository·develop·Indexed 26 days ago

https://github.com/element-hq/synapse

Synapse is a Matrix homeserver implementation. This documentation covers server-wide management via the Admin API, deployment using Docker and Docker Compose (including generic and federation sender workers), and integration testing with the Complement framework. It also provides guides for monitoring with Prometheus and Grafana, log analysis using lnav, and configuring JetBrains IDEs for PostgreSQL schema awareness.

Tokens
150.9K
Snippets
390
Records
1.2K
Agent score
89%

What's inside Synapse

  1. Overview of Synapse installation and maintenance

    develop

    Synapse is a Matrix homeserver implementation. To manage a production instance, you can perform the following tasks:

    • Installation & Configuration: Set up your instance and configure settings, including Single Sign-On (SSO).
    • Upgrades: Follow procedures to upgrade between Synapse versions.
    • Administration: Use the Admin API, install pluggable modules, or access the manhole.
    • Logging: Read log lines, configure standard logging, or set up structured logging.
    • Scaling: Scale the server by adding worker processes.
    • Monitoring: Set up metrics to monitor instance performance.
  2. Understand the Server Notices User Experience

    develop

    Server Notices use standard Matrix communication mechanisms and work with any Matrix client.

    User Behavior:

    • Invitations: When a user is first sent a notice, they receive an invitation to a room (the name is configurable via room_name). Users cannot reject this invitation; attempts to do so will result in an error.
    • Viewing Notices: Once accepted, notices appear in the room history, sent by the configured 'server notices user'.
    • Permissions: Users are prevented from sending messages in the notices room via power levels.
    • Leaving: Users can leave the room if they wish. If they have left, subsequent notices will trigger the creation of a new room.
  3. Understand Faster Joins (Partial Joins) in Synapse

    develop

    Synapse implements faster joins (also known as partial joins) to allow servers to request a lightweight response to the federation /send_join endpoint. This optimizes the joining process by providing a subset of the room state instead of the full state.

    A room is marked as partially joined in the database via the partial_state_rooms table, and specific events are marked as "partially stated" in the partial_state_events table.

    Key characteristics of a partially joined room:

    • Synapse uses a "best-effort approximation" of the room state.
    • Authorization check 5 (rules based on state before the event) is performed against the resolution of the partial state and the event's auth events.
    • Authorization check 6 (soft-failures based on current room state) is disabled.
    • Events with partial state are not considered outliers.
  4. Understand Synapse Streams

    develop

    Synapse uses streams as append-only logs of facts to notify different parts of the application (or different workers) about database changes. Streams allow components to respond to events like new messages, account data changes, or device updates.

    Common stream examples include:

    • Events stream: Reports new events (PDUs) created by Synapse or accepted from other homeservers.
    • Account data stream: Reports changes to user account data.
    • To-device stream: Reports when a device has a new to-device message.

    For a full list of available streams, refer to the synapse.replication.tcp.streams module.

  5. Get started with Element Synapse

    develop

    Synapse is an open source Matrix homeserver implementation. It is primarily distributed via the Element Server Suite (ESS), which provides a full Matrix stack tailored to specific use cases.

    ESS editions include:

    • ESS Community: Free distribution for small-to-mid-scale, non-commercial community use.
    • ESS Pro: Commercial distribution for professional use.
    • ESS TI-M: A version of ESS Pro focused on German National Digital Health Agency (Gematik) requirements for TI-Messenger Pro and ePA.
  6. Understand Synapse logical databases

    develop

    Synapse supports splitting its datastore across multiple physical databases. The schema is organized into logical databases:

    • state: Stores Matrix room state (e.g., state_groups, their relationships, and contents).
    • main: Stores all other data.
    • common: Contains schema files for tables that must exist on all physical databases.
  7. Supported User Authentication Methods in Synapse

    develop

    Synapse provides several built-in methods for authenticating users. You can use these out-of-the-box or extend the system using custom pluggable authentication modules.

    Built-in authentication methods:

    • Username and password: Standard credential-based login.
    • Email address and password: Authentication using an email address as the identifier.
    • Single Sign-On (SSO): Support for the following protocols:
      • SAML
      • OpenID Connect (OIDC)
      • CAS
    • JSON Web Tokens (JWT): Token-based authentication.
    • Administrator's shared secret: A method for administrative access.

    Extensibility: Synapse can be extended to support custom authentication schemes via optional "password auth provider" modules.

  8. Understand TCP Replication Protocol

    develop

    Synapse uses a TCP-based replication protocol to efficiently stream updates from a master process to workers, replacing older HTTP long-polling mechanisms. The protocol is line-based and uses a 'fire and forget' model.

    Key Protocol Characteristics:

    • Line-based: Each line starts with a command name followed by arguments.
    • Keep-alives: Both sides must send a command (like PING) at least every 5s. If no command is received within 15s, the connection is closed.
    • Reliability: The transport is generally unreliable (commands aren't resent if the connection drops), but RDATA commands include tokens that allow clients to resume streaming from the correct position upon reconnection.
    • Congestion Handling: If a client cannot consume messages fast enough, the server buffers a limited number of commands and then disconnects the client to prevent unbounded memory usage.
  9. Getting started with Synapse development

    develop

    Synapse is primarily written in Python. To contribute to the project:

    1. Setup: Follow the development environment setup guide.
    2. Workflow: Use the provided linters and testing suites to validate code.
    3. Issues: Search the GitHub issue tracker for bugs or features, specifically looking for the good first issue label if you are new.
    4. Advanced Topics: Familiarize yourself with database schema migrations, federation, and the internal build process.
    5. Git Hygiene: Follow the project's guidelines for maintaining a clean git history.
  10. Implement SQL delta files for database upgrades

    develop

    Delta files define the steps to upgrade the database. SQL deltas are used for simple schema changes.

    Requirements:

    • Naming: Files must be named *.sql, *.sql.postgres, or *.sql.sqlite.
    • Ordering: Files are applied in alphanumeric order. Use a prefix like 01, 02, etc. (e.g., 01add_bar_to_foo.sql).
    • Location: Place files in synapse/storage/schema/<database>/delta/<version>/.
    • Limitation: The SQL parser is simple. It does not support complex statements requiring a semicolon in the middle (like CREATE TRIGGER). Use Python delta files for such cases.
  11. Generate a fresh Synapse configuration file using Docker

    develop

    To generate a new homeserver.yaml and the necessary signing keys, use the generate command via docker-compose. You must provide the SYNAPSE_SERVER_NAME and SYNAPSE_REPORT_STATS environment variables.

    After generation, you should customize the resulting configuration file before starting the server.

    docker-compose run --rm -e SYNAPSE_SERVER_NAME=my.matrix.host -e SYNAPSE_REPORT_STATS=yes synapse generate