repository·main·Indexed 26 days ago
https://github.com/vlcn-io/cr-sqliteA loadable SQLite extension that implements Conflict-free Replicated Data Types (CRDTs) for synchronization. It provides functionality to create Convergent Replicated Records (CRR), manage database versions, and handle binary column packing for optimized data transport. The extension includes tools for backfilling tables with clock values, managing extension state via crsql_ExtData, and implementing Fractindex ordering through specialized views and triggers.
To synchronize data between two databases, you can extract changes from one database's crsql_changes table and insert them into the target database's crsql_changes table. This allows the target database to replay the changes and reach consistency.
# Logic for merging changes from one DB to another
changes = merge_from.execute("SELECT * FROM crsql_changes WHERE db_version = ?", (target_version,))
for change in changes:
merge_to.execute("INSERT INTO crsql_changes VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)", change)
merge_to.commit()Before using a table with CR-SQLite (Conflict-free Replicated Data Types), ensure it meets the following requirements via is_table_compatible. If a table fails these checks, it may cause replication conflicts or schema versioning issues.
Requirements:
NOT NULL.AUTOINCREMENT. Use a stable identity like a UUIDv7 to prevent primary key collisions across different nodes.NOT NULL column that is not part of the primary key must have a DEFAULT value to ensure forward and backward compatibility between schema versions.The bind_package_to_stmt function allows you to take a vector of ColumnValue enums and bind them to a prepared SQLite statement starting at a specific parameter offset.
Parameters:
stmt: A pointer to a sqlite::stmt.values: A reference to a Vec<ColumnValue> containing the data to bind.offset: The starting index for binding (0-indexed, but the function internally adjusts for SQLite's 1-based indexing).crsql_fill_db_version_if_needed to ensure the database version is correctly loaded into the extension's internal state. This function checks the SQLite PRAGMA data_version and, if necessary, fetches the current database version from the internal storage. This is typically used during initialization or when the database state might have changed externally.The get_or_create_key_for_insert function is used during row insertions to handle key retrieval or creation in a single step using INSERT OR IGNORE ... RETURNING logic.
Parameters:
db: A pointer to the sqlite3 database connection.pks: A slice of raw *mut value representing the primary key values.Returns:
Result<(bool, sqlite::int64), ResultCode>: A tuple where the boolean indicates if the row already existed (true) or was newly created (false), and the second element is the __crsql_key.pub fn get_or_create_key_for_insert(
&self,
db: *mut sqlite3,
pks: &[*mut value],
) -> Result<(bool, sqlite::int64), ResultCode>When working with the CR-SQLite extension at a low level (e.g., via FFI), you can manage extension-specific state using crsql_ExtData. This data structure holds schema versions, database versions, site IDs, and prepared statements used for internal synchronization operations.
Key lifecycle functions:
crsql_newExtData: Creates a new extension data structure for a given database and site ID buffer.crsql_freeExtData: Frees the allocated extension data.crsql_finalize: Finalizes the extension data.Note that INSERT_SENTINEL and DELETE_SENTINEL both use the value "-1" to represent these operations in the change log.
extern "C" {
pub fn crsql_fetchPragmaSchemaVersion(
db: *mut sqlite::sqlite3,
pExtData: *mut crsql_ExtData,
which: c_int,
) -> c_int;
pub fn crsql_fetchPragmaDataVersion(
db: *mut sqlite::sqlite3,
pExtData: *mut crsql_ExtData,
) -> c_int;
pub fn crsql_newExtData(
db: *mut sqlite::sqlite3,
siteIdBuffer: *mut c_char,
) -> *mut crsql_ExtData;
pub fn crsql_freeExtData(pExtData: *mut crsql_ExtData);
pub fn crsql_finalize(pExtData: *mut crsql_ExtData);
}is_crr function determines if a specific table in a SQLite database has been upgraded to a Conflict-free Replicated Data Type (CRR) by checking for the existence of the internal __crsql_itrig trigger.Use the connect function to establish a connection to a SQLite database file and automatically load the crsqlite extension. This enables CR-SQLite specific features like conflict-free replicated data types (CRDTs).
Note: The extension path is currently hardcoded to ../../core/dist/crsqlite within this package.
To enable Fractindex ordering for a specific table, use create_fract_view_and_view_triggers. This function automates the creation of a specialized view (suffixed with _fractindex) and the necessary INSTEAD OF triggers for INSERT and UPDATE operations. These triggers ensure that the Fractindex order is correctly maintained and that collisions are handled automatically.
Parameters:
db: A pointer to the sqlite3 database connection.table: The name of the base table to which the view and triggers will be applied.order_by_column: A pointer to a sqlite_nostd::value representing the column used for ordering.collection_columns: A vector of string slices (&str) representing the columns that define a unique collection (used for collision detection).crsql_as_crr('table_name') function. This converts the specified table into a CRR table, allowing it to track changes and support synchronization.crsql_finalize() to clean up or finalize the CRR state of a database connection, which is often used during teardown or after significant synchronization operations.