wa-sqlite

wa-sqlite is a WebAssembly build of SQLite that allows developers to write SQLite virtual filesystems (VFS) completely in JavaScript. This enables alternative browser storage options like IndexedDB and Origin Private File System (OPFS) for persistent SQLite storage in web applications.

Key features:

• Custom VFS Support: Use JavaScript to implement storage backends (e.g., IndexedDB, OPFS).
• Build Options: Supports both synchronous and asynchronous builds. Asynchronous builds (using Asyncify or JSPI) are required if you intend to use asynchronous extensions.
• Pre-built Artifacts: Most users can use the pre-built files located in the ./dist directory rather than building from source.

OPFSAdaptiveVFS maps OPFS access handles to VFS methods while supporting multiple connections through lazy handle management.

Key Features:

• Lazy Connection Handling: Since the OPFS spec typically allows only one open access handle per file, this VFS lazily closes the handle on one connection only when another connection requires it.
• Spec-Aware Optimization: On browsers that support multiple open access handles for a single file, it uses that capability to allow overlapping read transactions with write transactions.

Limitations:

• If multiple open access handles are not supported by the browser, only the following journaling modes are allowed: delete (default), memory, and off.

OPFSWriteAheadVFS is a synchronous OPFS VFS that implements its own write-ahead logging.

Key Features:

• Internal WAL: Write-ahead logging is implemented entirely within the VFS and is always on. It does not use the built-in SQLite WAL feature.

Requirements/Limitations:

• Browser Support: Requires the proposed readwrite-unsafe locking mode for OPFS access handles (available on Chromium browsers as of June 2024).

IDBMirrorVFS keeps all files in memory and persists them to IndexedDB. It works in all contexts.

Key Features:

• Performance: Faster than IDBBatchAtomicVFS both with and without contention.
• Durability Tuning: Can trade durability for performance by setting PRAGMA synchronous=normal.

Limitations:

• Memory Constraint: It can only be used with databases that fit within the available system memory.
• Page Size: Changing the page size after the database is created is not supported.

AccessHandlePoolVFS is an OPFS VFS designed for synchronous WebAssembly builds, offering significant performance advantages.

Key Features:

• Synchronous API: All methods are synchronous (do not return Promises).
• Optimized Locking: Because it restricts operation to a single wa-sqlite instance (no multiple connections), you can use PRAGMA locking_mode=exclusive, which enables PRAGMA journal_mode=wal to reduce write transaction overhead.

Limitations:

• Single Connection Only: Multiple connections are not supported.
• Not Filesystem Transparent: Database files in OPFS cannot be directly imported or exported.

OPFSCoopSyncVFS is a synchronous OPFS VFS that supports multiple connections and is filesystem transparent.

Key Features:

• Filesystem Transparency: Unlike AccessHandlePoolVFS, it is filesystem transparent.
• Hybrid Handle Management: Uses an access handle pool for most files, but uses lazy closing for shared files (the main database and journal) to support multiple connections.
• Synchronous Wrapper: To maintain a synchronous API, it uses a library wrapper that internally handles asynchronous errors (e.g., during locking) by waiting and retrying.

Limitations:

• Transaction Restriction: Transactions that access more than one main (non-temporary) database are not supported.

IDBBatchAtomicVFS stores database pages in IndexedDB and is suitable for all contexts (Window, Worker, SharedWorker, etc.).

Key Features:

• Batch Atomic Writes: Leverages IndexedDB to guarantee that an arbitrary set of changes is made completely or not at all. This allows for improved performance as an external journal file is not needed (the journal is kept in the page cache instead).
• Performance Tuning: You can trade durability for performance by setting PRAGMA synchronous=normal.

Limitations:

• Changing the page size after the database is created is not supported.
• Requires the cache size to be large enough to hold the journal to benefit from batch atomic mode.

OPFSAnyContextVFS uses the File and FileSystemWritableFileStream OPFS APIs, allowing it to run in any context (not just dedicated Workers).

Best Use Case:

• Recommended for read-only or nearly read-only databases.

Limitations:

• Write Performance: Write performance is very poor and degrades as the file grows.

The hello.js demo script can be executed in either a Window or a Worker context. By default, it runs in the Window context. To switch to a Worker context, append the worker query parameter to the URL in your browser.

Example:

• Window (default): https://example.com/demo/hello/index.html
• Worker: https://example.com/demo/hello/index.html?worker

The hello.js demo uses specific builds and Virtual File Systems (VFS) by default. You can experiment with different combinations by modifying the imports at the top of the hello.js file.

Default Configuration:

• Build: Asyncify
• VFS: IDBBatchAtomicVFS

Important Compatibility Constraints (as of May 2024):

• JSPI Build: Only works on recent Chromium browsers with an experiment flag enabled.
• OPFSWriteAheadVFS: Only works on recent Chromium browsers; requires FileSystemSyncAccessHandle with "readwrite-unsafe" locking.
• OPFS VFS Classes: These must be used within a Worker context.
• Asynchronous Builds: Some VFS classes require an asynchronous build (either Asyncify or JSPI) to function.

To run the official demo from the source tree:

1. Start the development server:

   yarn start

2. Open your browser to: http://localhost:8000/demo/?build=asyncify&config=IDBBatchAtomicVFS&reset

Demo Configuration via URL Parameters

You can control the demo behavior using the following query parameters: