Electric SQL Documentation
repository·main·Indexed 27 days ago
https://github.com/electric-sql/electricElectric is a real-time sync engine for Postgres that synchronizes data to clients via a low-level HTTP API, supporting partial replication through 'Shapes'. The project includes client libraries for TypeScript and Elixir, React integrations, a Durable Streams server implemented in Rust for persistent event streams, and the Electric Agents Desktop application.
What's inside Electric
- PGlite is a Postgres distribution compiled to WebAssembly (WASM) and packaged for JavaScript environments. It is designed to run as an embedded database in browsers, Node.js, Bun, and Deno.
Overview of Electric Products
mainElectric provides a data platform for multi-agent systems built on Postgres and Durable Streams. The platform consists of three primary products:
- Electric Agents: A runtime for long-lived agents. Agents are durable, synced entities that are resumable across devices, observable across teams, and forkable for experimentation.
- Electric Streams: A data primitive for the agent loop, providing persistent, addressable, real-time streams for agent session data.
- Electric Sync: The core sync engine technology. Specifically, Postgres Sync acts as the read-path sync engine for Postgres, managing partial replication, data delivery, and fan-out from Postgres to clients via HTTP.
Overview of the TypeScript client primitives
mainThe
@electric-sql/clientpackage provides two primary primitives for managing data synchronization:- ShapeStream: A primitive used to subscribe to a continuous stream of changes.
- Shape: A primitive used to retrieve the entire state of a shape whenever it changes.
Overview of Hosted Durable Streams
mainHosted Durable Streams on Electric Cloud provide persistent, resumable event streams over HTTP. They are designed for multi-agent and multi-user coordination (Durable Sessions), allowing shared logs that all participants can read, resume, and react to.
Key features include:
- Scalable Reads: Served via Electric Cloud's Sync CDN (tested up to 1M concurrent connections per stream).
- High Throughput: Supports ~240K writes/second for small messages and 15-25 MB/sec sustained throughput.
- Multi-language Support: Client libraries available for TypeScript, Python, Go, Rust, Java, Swift, PHP, Ruby, Elixir, and .NET.
- Pricing: Reads are free; includes 5 million writes/month for free.
Overview of Electric Sync Engine
mainElectric is a Postgres sync engine that enables real-time partial replication of Postgres data into local applications and services. It allows developers to transition from traditional data fetching to a data sync model, enabling instant, real-time, local-first experiences without needing to build a custom sync engine or modify existing Postgres data models, extensions (likepgvectororPostGIS), or migrations.Overview of ElectricSQL
mainElectricSQL is a realtime sync service for Postgres. It allows you to sync specific subsets of your Postgres data into local applications and services, enabling local-first development patterns.
Key offerings include:
- Electric: Sync subsets of Postgres data into local apps and services.
- Cloud: A hosted sync service designed to scale to millions of users.
- PGlite: Sync data into a lightweight WASM Postgres with real-time, reactive bindings.
Overview of TanStack DB
mainTanStack DB is a reactive, client-first data store designed for building high-performance applications. It provides sub-millisecond reactivity and instant local writes by acting as a client-side data layer. It is built on a TypeScript implementation of differential dataflow (
d2ts).Key benefits include:
- Avoiding network waterfalls and endpoint sprawl.
- Optimizing client-side re-rendering and performance.
- Taking the network off the interaction path for instantaneous UI updates.
- Enabling query-driven sync when used with Postgres Sync.
Overview of Postgres Sync
mainPostgres Sync (formerly known as Electric) is a read-path sync engine for Postgres. It handles partial replication, data delivery, and fan-out by connecting to your Postgres database via a
DATABASE_URL, consuming the logical replication stream, and distributing data into Shapes for client consumption.Key characteristics:
- Architecture: It is an Elixir application that runs as a separate service between your API and your database.
- Scalability: Clients consume data over an HTTP API that is compatible with CDNs, allowing for millions of concurrent users with minimal load on the primary database.
- Data Delivery: The same shape log is delivered to all subscribers (web, mobile, server workers, etc.) in real time with consistent ordering and guarantees.
Overview of Electric Sync
mainElectric Sync is a read-path sync engine for Postgres. It synchronizes data from Postgres into local clients over HTTP using a primitive called a Shape. It is designed for building fast, modern applications and multi-agent systems.Overview of Electric Next
mainElectric Next is a rebuilt sync engine that serves as the foundation for ElectricSQL. It provides an HTTP API for syncing Shapes of data from Postgres.
Developers can interact with Electric Next through:
- The direct HTTP API.
- Official client libraries (e.g., TypeScript).
- Official integrations (e.g., React).
- Custom client implementations in any language.
Overview of ElectricTelemetry modules
mainThe
electric_telemetrylibrary is organized into several functional modules:ElectricTelemetry: Provides basic utilities, including user option validation.ElectricTelemetry.ApplicationTelemetry: Defines metrics and periodic measurements for the entire BEAM runtime.ElectricTelemetry.StackTelemetry: Defines metrics specific to an Electric stack (e.g., shape stats, replication client stats). Note that no built-in measurements are defined here by default.- Reporter modules: Enabled individually to export metrics to specific destinations.
Understand Reservation techniques in Rich-CRDTs
mainRich-CRDTs use 'Reservations' to enable concurrent operations on a single data type by reserving specific amounts of resources for each client. There are three primary methods for implementing reservations:
- Escrow reservations: Resources are shared across a distributed cluster. Individual nodes possess a share of 'rights' to perform operations (e.g., a bounded counter where each node is allocated a specific number of tickets to sell).
- Escrow plus algorithm: Resources are allocated via an algorithm to maintain invariants (e.g., reserving specific nodes in a tree structure during a move operation to prevent cycles while allowing concurrent operations on other subtrees).
- Locking (mutual exclusion): Only the node holding the lock can access a resource. This is used for operations requiring global sequentiality, such as generating global sequential identifiers without gaps.