Electric SQL Documentation

repository·main·Indexed 27 days ago

https://github.com/electric-sql/electric

Electric is a real-time sync engine for Postgres that synchronizes data to clients via a low-level HTTP API, supporting partial replication through 'Shapes'. The project includes client libraries for TypeScript and Elixir, React integrations, a Durable Streams server implemented in Rust for persistent event streams, and the Electric Agents Desktop application.

Tokens
206.3K
Snippets
509
Records
1K
Agent score
94%

What's inside Electric

  1. Overview of Electric Products

    main

    Electric provides a data platform for multi-agent systems built on Postgres and Durable Streams. The platform consists of three primary products:

    • Electric Agents: A runtime for long-lived agents. Agents are durable, synced entities that are resumable across devices, observable across teams, and forkable for experimentation.
    • Electric Streams: A data primitive for the agent loop, providing persistent, addressable, real-time streams for agent session data.
    • Electric Sync: The core sync engine technology. Specifically, Postgres Sync acts as the read-path sync engine for Postgres, managing partial replication, data delivery, and fan-out from Postgres to clients via HTTP.
  2. Overview of Hosted Durable Streams

    main

    Hosted Durable Streams on Electric Cloud provide persistent, resumable event streams over HTTP. They are designed for multi-agent and multi-user coordination (Durable Sessions), allowing shared logs that all participants can read, resume, and react to.

    Key features include:

    • Scalable Reads: Served via Electric Cloud's Sync CDN (tested up to 1M concurrent connections per stream).
    • High Throughput: Supports ~240K writes/second for small messages and 15-25 MB/sec sustained throughput.
    • Multi-language Support: Client libraries available for TypeScript, Python, Go, Rust, Java, Swift, PHP, Ruby, Elixir, and .NET.
    • Pricing: Reads are free; includes 5 million writes/month for free.
  3. Overview of Electric Sync Engine

    main
    Electric is a Postgres sync engine that enables real-time partial replication of Postgres data into local applications and services. It allows developers to transition from traditional data fetching to a data sync model, enabling instant, real-time, local-first experiences without needing to build a custom sync engine or modify existing Postgres data models, extensions (like pgvector or PostGIS), or migrations.
  4. Overview of ElectricSQL

    main

    ElectricSQL is a realtime sync service for Postgres. It allows you to sync specific subsets of your Postgres data into local applications and services, enabling local-first development patterns.

    Key offerings include:

    • Electric: Sync subsets of Postgres data into local apps and services.
    • Cloud: A hosted sync service designed to scale to millions of users.
    • PGlite: Sync data into a lightweight WASM Postgres with real-time, reactive bindings.
  5. Overview of TanStack DB

    main

    TanStack DB is a reactive, client-first data store designed for building high-performance applications. It provides sub-millisecond reactivity and instant local writes by acting as a client-side data layer. It is built on a TypeScript implementation of differential dataflow (d2ts).

    Key benefits include:

    • Avoiding network waterfalls and endpoint sprawl.
    • Optimizing client-side re-rendering and performance.
    • Taking the network off the interaction path for instantaneous UI updates.
    • Enabling query-driven sync when used with Postgres Sync.
  6. Overview of Postgres Sync

    main

    Postgres Sync (formerly known as Electric) is a read-path sync engine for Postgres. It handles partial replication, data delivery, and fan-out by connecting to your Postgres database via a DATABASE_URL, consuming the logical replication stream, and distributing data into Shapes for client consumption.

    Key characteristics:

    • Architecture: It is an Elixir application that runs as a separate service between your API and your database.
    • Scalability: Clients consume data over an HTTP API that is compatible with CDNs, allowing for millions of concurrent users with minimal load on the primary database.
    • Data Delivery: The same shape log is delivered to all subscribers (web, mobile, server workers, etc.) in real time with consistent ordering and guarantees.
  7. Overview of ElectricTelemetry modules

    main

    The electric_telemetry library is organized into several functional modules:

    • ElectricTelemetry: Provides basic utilities, including user option validation.
    • ElectricTelemetry.ApplicationTelemetry: Defines metrics and periodic measurements for the entire BEAM runtime.
    • ElectricTelemetry.StackTelemetry: Defines metrics specific to an Electric stack (e.g., shape stats, replication client stats). Note that no built-in measurements are defined here by default.
    • Reporter modules: Enabled individually to export metrics to specific destinations.
  8. Understand Reservation techniques in Rich-CRDTs

    main

    Rich-CRDTs use 'Reservations' to enable concurrent operations on a single data type by reserving specific amounts of resources for each client. There are three primary methods for implementing reservations:

    1. Escrow reservations: Resources are shared across a distributed cluster. Individual nodes possess a share of 'rights' to perform operations (e.g., a bounded counter where each node is allocated a specific number of tickets to sell).
    2. Escrow plus algorithm: Resources are allocated via an algorithm to maintain invariants (e.g., reserving specific nodes in a tree structure during a move operation to prevent cycles while allowing concurrent operations on other subtrees).
    3. Locking (mutual exclusion): Only the node holding the lock can access a resource. This is used for operations requiring global sequentiality, such as generating global sequential identifiers without gaps.