declarative-sqlite
website·Indexed 31 days ago
https://declarative-sqlite.linden.no/A TypeScript library for offline-first browser apps that runs SQLite (WebAssembly, OPFS) with a declarative schema. It features live queries, automatic migrations, a sync runtime for server synchronization, and a dedicated React layer via declarative-sqlite/react.
What's inside declarative-sqlite
- declarative-sqlite is a TypeScript library designed for browser-based apps (PWAs, web views) that require offline functionality. It runs a real SQLite database in the browser using WebAssembly and stores data in the Origin Private File System (OPFS) where available. The library manages a local copy of server data, allows offline edits, and synchronizes those edits back to the server when connectivity is restored.
Understand automatic migration behavior in declarative-sqlite
declarative-sqlite does not use migration files. Instead,
Database.opencompares the current schema with the database on disk and applies differences in a single transaction.Automatic behaviors include:
- New tables: Created via
CREATE TABLEincluding indexes. - New columns: Added via
ALTER TABLE … ADD COLUMN. If the column isnotNull, existing rows receive the default value. - Indexes/Unique keys: New ones are created. If an index exists with the same name but different columns or type, it is dropped and recreated.
- Removals: Tables or columns removed from the schema are not deleted from the database to prevent data loss. They are reported as 'extra' in the migration plan.
- New tables: Created via
Handle server-side change notifications (Ticks)
When the server notifies the client of a change (via WebSockets, SSE, etc.), pass the notification to the runtime. The runtime will pull the affected scopes using
from: 'window'.To optimize, register which scopes are currently active/on-screen so the runtime knows which ones to pull.
// Pass server notification to runtime socket.on('TableChanged', (msg) => { sync.ticks.notify({ table: 'task', seq: msg.seq, scopes: msg.projectIds }); }); // Register an active scope const unregister = sync.pull.registerScope('task', { project_id: 42 }); // Call unregister() when the view closes // Force immediate resolution of pending ticks sync.ticks.flush();Initialize storage using openAdapter
The
openAdapterfunction automatically selects the best available storage backend based on browser support. It attempts to use backends in the following priority order:- opfs: Used if the browser supports OPFS sync access handles (Chrome/Edge 108+, Firefox 111+, Safari 17+). Persistent.
- indexeddb: Used if OPFS is missing or fails (e.g., Safari 16). Persistent, but with performance and data loss caveats.
- memory: Used as a final fallback (e.g., private windows). Not persistent; data is lost on reload.
The returned
backendstring indicates which storage was selected, andwarningsprovides details on any fallbacks that occurred.import { openAdapter } from 'declarative-sqlite'; const { adapter, backend, warnings } = await openAdapter({ name: 'app.db' });Test synchronization behavior using MemoryAdapter and FakeTransport
Because the library runs against a real SQLite engine, synchronization behavior can be tested with standard unit tests without requiring a browser or server. UseMemoryAdapterfor an in-memory SQLite database (Node.js) andFakeTransportto simulate a server following the server protocol.import { describe, it, expect } from 'vitest'; import { Database, MemoryAdapter, FakeTransport, createSyncRuntime } from 'declarative-sqlite'; import { appSchema } from '../src/schema'; describe('task sync', () => { it('keeps the user edit when another device changes the same row', async () => { // 1. Setup in-memory DB and fake server const db = await Database.open({ schema: appSchema, adapter: new MemoryAdapter() }); const server = new FakeTransport(); // Seed server with initial data (use uppercase wire names for tables/columns) server.seed('TASK', [{ id: 'A', data: { PROJECT_ID: 42, TITLE: 'Pump', HOURS: 1 } }]); // 2. Initialize sync runtime const sync = await createSyncRuntime({ db, transport: server, deviceId: 'test', debounceMs: 0 }); await sync.pull.pull('task', { project_id: 42 }); // 3. Simulate local change and remote server edit await sync.outbox.record({ table: 'task', systemId: 'A', changes: { hours: 5 } }); server.serverEdit('TASK', 'A', { TITLE: 'Pump (renamed)' }); await sync.pull.pull('task', { project_id: 42 }); // 4. Assert results const row = await db.tables.task.get('A'); expect(row).toMatchObject({ hours: 5, title: 'Pump (renamed)' }); await sync.push.pushNow(); expect(await sync.outbox.counts()).toMatchObject({ pending: 0, rejected: 0 }); // 5. Cleanup sync.close(); await db.close(); }); });Steps to upgrade declarative-sqlite to v3
Follow these steps to port your application to v3:
- Update Schema: Remove
.lww(), mark server-owned tables with.synced(), and ensure everynotNullcolumn has a default value. - Update Setup: Replace the database initialization sequence with
openAdapter,Database.open, andcreateSyncRuntime. - Implement Transport: Create a
SyncTransportfor your API. The server must support sequence numbers and batch idempotency. - Update Writes: Replace direct writes to synced tables with
sync.outbox.record. Note that synced tables no longer have write methods ondb.tables. - Update UI/Streams: Replace RxJS streams with live queries and use
useDraftFieldfor form inputs.
- Update Schema: Remove
Install declarative-sqlite v3
To use the current version (v3) of the library, install it via npm.npm install declarative-sqliteCritical rules for writing code with declarative-sqlite
To avoid runtime errors and sync state corruption, follow these constraints:
- Installation: Use
npm install declarative-sqlitefor v3. Pin to2.xif the old API is required. - Synced Tables: Never write to a
.synced()table directly viadb.tablesorexecute. Usesync.outbox.record({ table, systemId, changes })to modify existing rows. Note thatoutbox.recordcannot create or delete rows; these must come from the server via pulling. - Inserts: Always supply a
system_id(e.g.,crypto.randomUUID()) when inserting into plain tables, as it is not auto-generated. - Schema Defaults: Every
.notNull()column must have a default value matching its type (e.g.,t.integer('n').notNull(0)ort.text('s').notNull('')). - Live Queries: In the
readsconfiguration of a live query, declare every table being read (including joined tables) and set thekeyto the row ID column (typicallysystem_id). - Resource Management: Call
query.close()for live queries andsync.close()before callingawait db.close(). - Architecture: Maintain exactly one database and one sync runtime per application, instantiated outside React components and provided via
<SyncProvider>. - UI Inputs: Use
useDraftFieldfor editable inputs bound to synced columns instead ofuseState+record. Convert string event values to numbers duringonChange. - Naming Conventions: Use lowercase names for local operations (e.g.,
sync.pull.pull('task', ...)). Use uppercase names for wire/transport operations (e.g.,FakeTransport.seed('TASK', ...)). - Reserved Names: Do not name tables
outboxorsync_cursor. Do not redefinesystem_id,system_removed, orsync_seqcolumns with different types. - Transport Errors: Throw exceptions from the transport on network failure. Do not return empty results, as thrown
pushoperations are retried with the same batch ID. - Testing: Use
MemoryAdapterandFakeTransportinstead of mocking the library.
- Installation: Use
Record local changes in the outbox
Edits to synced rows must be recorded via the outbox. This writes new values to the local row and creates outbox entries in one transaction. Columns provided in a single
recordcall form a change group and are always pushed in the same batch.Constraints:
- Table must be synced.
- Row must exist locally.
- Column must be in the schema.
changescannot be empty or exceed 500 columns.- Values cannot exceed 4000 JSON characters (
ValueTooLongError).
Note: Creating or deleting rows is not handled by the outbox; these operations must come from the server via a pull.
const groupId = await sync.outbox.record({ table: 'task', systemId: row.system_id, changes: { hours: 3.5, title: 'Install pump' }, });Plan a migration without opening the database
You can inspect the differences between your schema and the database on disk without executing the migration by using theplanMigrationfunction. This allows you to identify extra tables or columns and see the SQL operations that would be performed.import { planMigration } from 'declarative-sqlite'; await adapter.open(); const plan = await planMigration(adapter, schema); console.log(plan.diff.extraTables); // tables in the database but not the schema console.log(plan.diff.extraColumns); // columns in the database but not the schema console.log(plan.operations); // Array of { description, sql: [...] }Initialize declarative-sqlite in a React application
To usedeclarative-sqlite/react, open the database and sync runtime outside of the React component tree and provide them via theSyncProvider. TheSyncProvidermanages the lifecycle of open drafts, automatically saving them when the page is hidden, the provider unmounts, or therouteKeychanges. UseuseDatabase()anduseSyncRuntime()hooks to access these objects in child components.import { SyncProvider } from 'declarative-sqlite/react'; // Initialize outside React const db = await Database.open<Rows, 'task'>({ schema, adapter }); const sync = await createSyncRuntime({ db, transport, deviceId }); root.render( <SyncProvider db={db} sync={sync} routeKey={location.pathname}> <App /> </SyncProvider>, );Implement sequence numbering for data synchronization
The server must maintain a single increasing counter called the sequence number (
seq).- Every time a row is created, changed, or removed, it must be stamped with the next
seqvalue. - The client requests rows with a
seqgreater than the highest one it has already seen for a specific table and scope. - Consistency Note: Because sequence numbers may be assigned in a different order than transactions commit, the client may re-read the last 1000 sequence numbers upon receiving a change notification (
from: 'window'). The server does not need special logic to handle this; it simply fulfills the request for rows above the providedaftervalue.
- Every time a row is created, changed, or removed, it must be stamped with the next