Turbosql

repository·main·Indexed 42 days ago

https://github.com/trevyn/turbosql

A local data persistence layer for Rust backed by SQLite. It provides a type-safe API for CRUD operations using Rust structs and automates schema definition and migrations via the #[derive(Turbosql)] macro. Version 0.14.0 includes features such as the select! and execute! macros, support for WASM environments, and automatic migration management via migrations.toml.

Tokens
3K
Snippets
9
Records
12
Agent score
54%

What's inside Turbosql

  1. How Automatic Schema Migrations Work

    main

    Turbosql automates database schema management via the #[derive(Turbosql)] macro:

    1. Generation: At compile time, the macro generates a migrations.toml file in your project root describing the schema.
    2. Migration: When you change a struct, the macro generates new SQL migration statements and appends them to migrations.toml. These migrations are embedded in your binary.
    3. Application: On launch, the binary automatically applies any pending migrations in sequence to the existing database file.

    Important Migration Rules:

    • One-way Ratchet: Migrations are append-only. An older binary cannot operate on a database file that has been migrated to a newer schema.
    • Manual Edits: You can manually remove unused or reverted migrations from migrations.toml before release, but any database that has already applied them will error and require a rebuild.
    • Custom Migrations: You can manually add your own SQL migration entries (e.g., for creating indexes) to the bottom of the migrations_append_only list in migrations.toml.
  2. Using Turbosql in Async Environments

    main

    Because SQLite uses blocking synchronous APIs, you should run Turbosql operations within a thread pool using your executor's blocking task facility (e.g., tokio::task::spawn_blocking).

    Transactions in Async

    Turbosql uses persistent thread_local database connections. A continuous sequence of database calls from the same thread is guaranteed to use the same exclusive connection. This allows you to perform transactions by executing BEGIN IMMEDIATE TRANSACTION and COMMIT manually within a spawn_blocking block.

    #[tokio::main]
    async fn main() -> Result<(), Box<dyn std::error::Error>> {
        tokio::task::spawn_blocking(|| -> Result<(), turbosql::Error> {
            Person { rowid: None, age: Some(21) }.insert()?;
            execute!("BEGIN IMMEDIATE TRANSACTION")?;
            let p = select!(Person "WHERE rowid = ?", 1)?;
            execute!(
                "UPDATE person SET age = ? WHERE rowid = ?",
                p.age.unwrap_or_default() + 1,
                1
            )?;
            execute!("COMMIT")?;
            Ok(())
        }).await??;
        Ok(())
    }
  3. Basic Usage of Turbosql

    main

    Turbosql provides a high-level API for CRUD operations using Rust structs. To use it, derive Turbosql on your struct. Note that a rowid: Option<i64> member is required and enforced at compile time for all Turbosql structs.

    Common operations include:

    • .insert(): Creates a new row and returns the rowid.
    • select!(): Retrieves data using the select! macro.
    • .update(): Updates the database row with the current values in the struct (based on rowid).
    • execute!(): Runs manual SQL commands.
    use turbosql::{Turbosql, select, execute};
    
    #[derive(Turbosql, Default)]
    struct Person {
        rowid: Option<i64>, // rowid member required & enforced at compile time
        name: Option<String>,
        age: Option<i64>,
        image_jpg: Option<Vec<u8>>
    }
    
    fn main() -> Result<(), Box<dyn std::error::Error>> {
        let name = "Joe";
    
        // INSERT a row
        let rowid = Person {
            name: Some(name.to_string()),
            age: Some(42),
            ..Default::default()
        }.insert()?;
    
        // SELECT all rows
        let people = select!(Vec<Person>)?;
    
        // SELECT multiple rows with a predicate
        let people = select!(Vec<Person> "WHERE age > " 21)?;
    
        // SELECT a single row with a predicate
        let mut person = select!(Person "WHERE name = " name)?;
    
        // UPDATE based on rowid, rewrites all fields in database row
        person.age = Some(43);
        person.update()?;
    
        // UPDATE with manual SQL
        execute!("UPDATE person SET age = " 44 " WHERE name = " name)?;
    
        // DELETE
        execute!("DELETE FROM person WHERE rowid = " 1)?;
    
        Ok()
    }
  4. Locating the SQLite Database File

    main

    Turbosql uses the directories_next crate to determine the data directory. The database file is named {executable_name}.sqlite and is located in the project data directory:

    • Linux: $XDG_DATA_HOME/{exe_name} or $HOME/.local/share/{exe_name}
    • macOS: $HOME/Library/Application Support/{exe_name}
    • Windows: {FOLDERID_LocalAppData}\{exe_name}\data
  5. Reference: select! Macro Query Forms

    main

    The select! macro supports various return types and syntax patterns:

    Target TypeSyntax ExampleBehavior
    Primitiveselect!(String "SELECT name FROM person")?Returns one value; errors if no rows available.
    Primitive (Short)select!(String "name FROM person WHERE rowid = ?", id)?SELECT is optional; handles parameter binding.
    Vec<T>select!(Vec<String> "name FROM person")?Returns a Vec; returns empty Vec if no rows.
    Option<T>select!(Option<String> "name FROM person")?Returns Ok(None) if no rows; Error on actual error.
    Turbosql Structselect!(Person "WHERE name = ?", name)?Column list and table name are optional if using a Turbosql struct.
    Custom Structselect!(Vec<NameAndAdult> "name, age >= 18 AS adult FROM person")?Use other structs; column names must match and you must specify the source table.
  6. Use the Turbosql trait for CRUD operations

    main

    When you use #[derive(Turbosql)] on a struct, it implements the Turbosql trait, providing high-level methods for database interaction.

    Key requirements for rowid:

    • For insert operations (insert, insert_mut, insert_batch), the rowid field in your struct must be None.
    • For update and delete operations, the rowid field must be Some(id).

    Methods:

    • insert(&self) -> Result<i64, Error>: Inserts the row and returns the new rowid.
    • insert_mut(&mut self) -> Result<i64, Error>: Inserts the row and updates the struct's rowid field to match the database.
    • insert_batch<T: AsRef<Self>>(rows: &[T]) -> Result<(), Error>: Inserts multiple rows.
    • update(&self) -> Result<usize, Error>: Updates the existing row based on rowid. Returns the number of rows updated (should be 1).
    • update_batch<T: AsRef<Self>>(rows: &[T]) -> Result<(), Error>: Updates multiple rows.
    • delete(&self) -> Result<usize, Error>: Deletes the existing row based on rowid. Returns the number of rows deleted (should be 1).
    // Example usage of the Turbosql trait methods
    // (Assuming a struct `User` has `#[derive(Turbosql)]`)
    
    // 1. Insert a new user
    let new_user = User { rowid: None, name: "Alice".to_string() };
    let id = new_user.insert()?;
    
    // 2. Update an existing user
    let mut existing_user = User { rowid: Some(id), name: "Alice Smith".to_string() };
    existing_user.update()?;
    
    // 3. Delete a user
    existing_user.delete()?;
  7. Configure the database path with set_db_path

    main

    By default, Turbosql determines a database path based on your executable's name and stores it in the system's data directory. To use a specific file or location, call set_db_path before using any Turbosql macros or performing database operations.

    Note: You cannot change the path once the database has already been opened/initialized.

    use std::path::Path;
    
    // Set the path before any other Turbosql calls
    set_db_path(Path::new("my_custom_database.sqlite"))?;
  8. Get current time in milliseconds or microseconds

    main

    Turbosql provides two convenience functions for generating timestamps based on the UNIX epoch:

    • now_ms(): Returns current time as i64 milliseconds.
    • now_µs(): Returns current time as i64 microseconds.
    let timestamp = now_ms();
    // Use timestamp in your struct for creation dates
  9. Use Turbosql in WASM environments

    main

    When targeting wasm32, Turbosql re-exports the core API from turbosql_impl. You can use the Turbosql struct along with execute, select, and update functions to interact with the database. Note that now_ms() is currently not implemented for the wasm32 target and will cause a panic if called.

    // In a wasm32 target environment
    use turbosql::{Turbosql, execute, select, update};
    
    // Usage of re-exported symbols from turbosql_impl
  10. Perform a database checkpoint with checkpoint

    main

    Manually triggers a PRAGMA wal_checkpoint(PASSIVE). This helps manage the Write-Ahead Log (WAL) files. If no other threads have open connections, this operation can clean up the -wal and -shm files.

    Returns a CheckpointResult containing:

    • busy: Should always be 0.
    • log: Number of modified pages in the WAL.
    • checkpointed: Number of pages successfully moved back to the main database file.
    match checkpoint() {
        Ok(res) => println!("Checkpoint successful. Log size: {}", res.log),
        Err(e) => eprintln!("Checkpoint failed: {}", e),
    }
  11. Handle Turbosql errors

    main

    Turbosql uses a custom Error enum that wraps underlying library errors. When handling results, you may encounter:

    • Error::Rusqlite: Errors originating from the rusqlite driver (SQL syntax, constraint violations, etc.).
    • Error::SerdeJson: Errors during JSON serialization/deserialization.
    • Error::OtherError: Internal Turbosql errors (e.g., attempting to set a database path after initialization).
    #[derive(thiserror::Error, Debug)]
    pub enum Error {
    	#[error(transparent)]
    	Rusqlite(#[from] rusqlite::Error),
    	#[error(transparent)]
    	SerdeJson(#[from] serde_json::Error),
    	#[error("Turbosql Error: {0}")]
    	OtherError(&'static str),
    }