DOCSEARCHbyAlgolia
Submit Docs

absurd-sql

repository·master·Indexed 26 days ago

https://github.com/jlongster/absurd-sql

A persistent backend for sql.js that treats IndexedDB as a block-based disk, allowing SQLite databases to be stored and accessed in chunks in the browser. It includes the SQLiteFS class for filesystem interaction, an IndexedDBBackend for persistence, and a MemoryBackend for in-memory operations. The library requires a specific fork of sql.js (@jlongster/sql.js) and supports a fallback mode for browsers without SharedArrayBuffer.

Tokens
1.3K
Snippets
4
Records
11
Agent score
88%

What's inside absurd-sql

  1. Initialize absurd-sql on the main thread

    master

    Because absurd-sql must run in a worker, you need to initialize a backend handler on the main thread. This is specifically required for Safari to support nested workers by proxying worker creation through the main thread.

    Use initBackend from absurd-sql/dist/indexeddb-main-thread and pass your worker instance to it.

    import { initBackend } from 'absurd-sql/dist/indexeddb-main-thread';

function init() {
  let worker = new Worker(new URL('./index.worker.js', import.meta.url));
  // This is only required because Safari doesn't support nested
  // workers. This installs a handler that will proxy creating web
  // workers through the main thread
  initBackend(worker);
}

init();

  3. Configure the SQLite IndexedDB backend in a worker

    master

    In your worker file, you must initialize sql.js, create a SQLiteFS using an IndexedDBBackend, and register it with the SQL instance. You then mount the filesystem to a path (e.g., /sql) and open a database file within that path.

    Note on Fallback Mode: If SharedArrayBuffer is not available (e.g., in Safari), you must manually read the file contents using readIfFallback() before opening the database to ensure data is loaded correctly.

    import initSqlJs from '@jlongster/sql.js';
import { SQLiteFS } from 'absurd-sql';
import IndexedDBBackend from 'absurd-sql/dist/indexeddb-backend';

async function run() {
  let SQL = await initSqlJs({ locateFile: file => file });
  let sqlFS = new SQLiteFS(SQL.FS, new IndexedDBBackend());
  SQL.register_for_idb(sqlFS);

  SQL.FS.mkdir('/sql');
  SQL.FS.mount(sqlFS, {}, '/sql');

  const path = '/sql/db.sqlite';
  if (typeof SharedArrayBuffer === 'undefined') {
    let stream = SQL.FS.open(path, 'a+');
    await stream.node.contents.readIfFallback();
    SQL.FS.close(stream);
  }

  let db = new SQL.Database(path, { filename: true });
  db.exec(`
    PRAGMA journal_mode=MEMORY;
  `);

   // Your code
}

  4. Understand Fallback Mode limitations

    master

    In browsers without SharedArrayBuffer (such as Safari), absurd-sql operates in a fallback mode.

    Limitations:

    • Only one tab can write to the database at a time.
    • If multiple tabs attempt to write simultaneously, the operation will throw an error to prevent database corruption.

  5. Configure server headers for SharedArrayBuffer

    master

    To enable SharedArrayBuffer and the Atomics API, your server must respond with the following security headers to isolate the process:

    • Cross-Origin-Opener-Policy: same-origin
    • Cross-Origin-Embedder-Policy: require-corp

  6. Verify write safety with isSafeToWrite()

    master

    Use isSafeToWrite(localData, diskData) to determine if it is safe to write in-memory data to disk. The function compares a specific byte range (indices 24 to 39) between the localData and diskData to detect if the underlying file has changed underneath the application.

    • Returns true if the byte ranges match or if both inputs are null.
    • Returns false if the byte ranges differ or if only one input is null.

  8. Use MemoryBackend for in-memory file operations

    master
    The MemoryBackend class provides an in-memory implementation of file operations, useful for testing or scenarios where persistent storage is not required. You can initialize it with existing file data and manage files using createFile and getFile.