ncruces/go-sqlite3

The repository is organized into several packages depending on your needs:

• github.com/ncruces/go-sqlite3: Provides direct access to the C SQLite API.
• github.com/ncruces/go-sqlite3/driver: Provides a database/sql compatible driver.
• github.com/ncruces/go-sqlite3/vfs: Wraps the C SQLite VFS API and provides a pure Go implementation of the Virtual File System.
• github.com/ncruces/go-sqlite3/gormlite: Provides a driver specifically for GORM.

This package provides a pure Go implementation of the SQLite OS Interface (VFS), replacing the default SQLite VFS. It exposes interfaces that allow developers to implement custom VFSes.

Key Differences from standard SQLite:

• File Locking: Uses OFD locks on Linux/macOS (requires Linux 3.15+) and LockFileEx on Windows. It can also use BSD locks or dot-file locks via build tags.
• WAL Mode: Uses mmap on Unix and MapViewOfFile on Windows for shared-memory WAL-index. If these are not supported, WAL support is limited and requires EXCLUSIVE locking mode.
• Compatibility: The default configuration is compatible with standard Unix and Windows SQLite VFSes. However, using incompatible VFSes concurrently on the same database will cause data corruption.

This module behaves similarly to SQLite in multi-thread mode:

1. database/sql API: Safe to use concurrently according to standard Go documentation.
2. Direct Connections: The module is goroutine-safe, provided that no single database connection (or any object derived from it) is used concurrently by multiple goroutines.

The reader VFS allows you to treat any object implementing the io.ReaderAt interface as an immutable SQLite database. This is useful for accessing database files that are provided as streams or in-memory buffers without needing to write them to a physical file on disk first.

// Example concept: any io.ReaderAt can be used as an immutable database
// via the reader VFS.
var myReader io.ReaderAt = ... 
// Use myReader with the reader VFS to open the database.

The adiantum VFS uses a specific cipher composition for disk encryption:

• Stream Cipher: XChaCha12
• Block Cipher: AES
• Hashing: NH and Poly1305
• Key Derivation: Argon2id (derives 256-bit keys from plain text)
• Block Size: 4 KiB

Note: Adiantum is designed for disk encryption where the threat model assumes an adversary can read multiple snapshots of a disk. The primary security property is that an adversary can only determine if data in a sector has changed or not.

When using this module, be aware of the following:

• VFS Implementation: This module replaces the standard SQLite OS Interface (VFS) with a pure Go implementation. This has specific design trade-offs.
• Memory Usage: Because each database connection executes within a Wasm sandboxed environment, memory usage will be higher than other SQLite drivers.
• Performance: The database/sql driver performance is competitive with other alternatives, and the Wasm/VFS layers are benchmarked against SQLite's own speedtest1.

The mvcc (Multi-Version Concurrency Control) VFS is an EXPERIMENTAL in-memory SQLite implementation. It is designed to provide several advantages over the standard memdb VFS:

• Isolation from Panics: Panics in one connection do not corrupt the shared database.
• Non-blocking Concurrency: A single writer does not block readers, and readers never block each other or the writer.
• Instant Snapshots: Allows for immediate creation of database snapshots.

Use this VFS when you need high concurrency or the ability to create multiple independent copies of a database state quickly.

Users of the xts VFS should be aware of the following security constraints:

• No Tamper Protection: This package does not protect against database tampering or forgery. It does not use page-level MACs (Message Authentication Codes).
• Backup Forgery: Because the encryption is deterministic, an adversary can identify changes between snapshots. If you are keeping xts encrypted backups and require protection against forgery, you must sign your backups and verify those signatures before restoring them.
• Alternative: For significantly better performance and potentially improved security via a "wide-block" cipher, consider using the adiantum VFS instead.

The package supports WAL mode using shared memory.

Using sqlite3_dotlk for WAL: If you use the sqlite3_dotlk build tag, you get a cross-platform, in-process memory sharing implementation. To avoid excessive concurrency with this implementation, set your connection pool limit using: db.SetMaxOpenConns(max(2, runtime.GOMAXPROCS(0)))

Limited WAL support: If your build does not support shared memory (check via vfs.SupportsSharedMemory), you must use EXCLUSIVE locking mode to create, read, and write WAL databases. When using database/sql in this mode, you must disable connection pooling with db.SetMaxOpenConns(1).

// Recommended connection limit for sqlite3_dotlk
db.SetMaxOpenConns(max(2, runtime.GOMAXPROCS(0)))