Hiqlite Documentation

Hiqlite is an embeddable SQLite database that forms a Raft cluster to provide:

• Strong consistency and high availability.
• Replication and automatic leader fail-over.
• Self-healing capabilities (nodes can recover from un-graceful shutdowns or full data volume loss).

Unlike traditional SQLite replication solutions that might require independent processes or specialized file systems, Hiqlite is built as an async wrapper around rusqlite and uses openraft for Raft logic, providing its own storage and network implementations. This allows you to keep data local for fast reads while maintaining the benefits of a distributed system.

The dlock feature provides distributed locks synchronized across all Raft nodes. This is useful when you need to perform complex logic (fetch, compute, write) that cannot be contained within a single SQL transaction due to Raft replication constraints.

Usage Pattern:

• Acquire a lock on a specific key.
• Perform your business logic.
• The lock is automatically released when the Lock object is dropped.

Important Limitation: To prevent deadlocks caused by network partitions or crashed nodes, distributed locks have a maximum lifetime of 10 seconds. If a lock is held longer than 10 seconds, it is considered "dead" and will be released by the system.

Hiqlite configuration (NodeConfig) can be managed via TOML files, environment variables (from_env()), or programmatically (from_toml()).

When adding new read-only replicas to a cluster, you may want to prevent them from automatically becoming voting Raft members during startup. To achieve this, set learner_only = true in your configuration or use the environment variable HQL_LEARNER_ONLY=true. This setting does not demote existing voters.

The listen_notify feature allows nodes to send real-time messages to each other, similar to Postgres' LISTEN/NOTIFY mechanism. It requires the cache feature.

Delivery Guarantees:

• Raft Members: If the node is a full Raft member, you get guaranteed once delivery for any listen() call.
• Remote Clients: If using a remote client connected to a cluster without local replicated state, it behaves like standard Postgres (messages are dropped if no one is currently listening).

Warning: When using notify() via the hiqlite::Client, you must ensure every node is actively consuming messages via listen(). Hiqlite uses an unbound channel internally; failing to consume messages can lead to the channel filling up.

To run Hiqlite as a standalone server instead of embedding it, you can install the binary with the server feature enabled.

1. Install the binary: Use cargo install hiqlite --features server.
2. Generate a configuration: It is recommended to start by generating a template config file using hiqlite generate-config -h.
3. Start a node: Use hiqlite serve -h.

Important Notes:

• The --node-id provided at startup must match a value defined in the HQL_NODES section of your configuration. Overwriting the node ID at startup allows you to reuse the same configuration file for multiple nodes.
• For testing without TLS, use the --insecure-cookie option and generate a testing password with -p.

# Install the server binary
cargo install hiqlite --features server

# Generate a template config
hiqlite generate-config -h

# Start a node
hiqlite serve -h

hiqlite-wal provides a Write-Ahead Log (WAL) implementation compatible with openraft. It uses memory-mapped files to minimize syscalls and is designed for high performance.

To use it, integrate it with openraft by calling LogStore::start().

Key characteristics:

• File Management: Instead of overwriting a single file, it creates new files on log roll-over. Files are pre-populated to ensure they have a fixed length, preventing disk relocation stress on SSDs.
• Flush Strategy: You can choose a "flush to disk" strategy based on your specific durability requirements.
• Generic Design: While optimized for Hiqlite, it is generic and works with any openraft implementation.

Hiqlite is highly modular via Cargo features. You can customize the build to include or exclude specific capabilities like SQLite storage, S3 backups, or distributed locking.

Default Features: When using the default feature set, the following are enabled:

• auto-heal
• backup
• sqlite
• toml

Common Feature Combinations:

• Full Suite: Use the full feature to enable everything except the standalone server mode.
• In-Memory KV Store: Disable all default features and only enable cache to run a replicated in-memory key-value store without requiring disk/SQLite.
• Standalone Server: Use the server feature to run Hiqlite as a standalone database cluster.

# Example: Enabling specific features in Cargo.toml
[dependencies]
hiqlite = {
    version = "0.14.0",
    features = ["cache", "dlock", "listen_notify"]
}

If you want to run Hiqlite as a standalone database/cluster rather than embedding it as a crate in your application, you can install the server binary via Cargo.

Note: Embedding Hiqlite as a crate is highly recommended over using the server mode, as embedding allows for local data access and significantly faster SELECT speeds by avoiding network round-trips.

cargo install hiqlite --features server

If you lose an entire cluster, you can perform disaster recovery by restoring from an encrypted S3 backup.

Steps to restore:

1. Ensure the cluster is shut down.
2. Set the HQL_BACKUP_RESTORE environment variable using one of these prefixes:
   • s3:<backup_file_name> (for encrypted backups on S3)
   • file:<path_to_sqlite_file> (for plain SQLite files on disk)
3. Start the cluster.
4. Crucial: Once the cluster has successfully restarted, remove the HQL_BACKUP_RESTORE environment variable to prevent accidental re-restores on subsequent boots.

# Example: Restoring from an S3 backup
export HQL_BACKUP_RESTORE="s3:my-encrypted-backup-file"
./start_cluster_command
# After successful start, unset the variable
unset HQL_BACKUP_RESTORE