Extralite

repository·main·Indexed 43 days ago

https://github.com/digital-fabric/extralite

A high-performance SQLite wrapper for Ruby 3.0+ featuring multiple query modes (hash, array, splat), concurrency support for multi-threaded and multi-fibered applications, and advanced parameter binding. It includes support for prepared queries, batch execution, structured transforms for complex object graphs, and fine-grained transaction control via savepoints.

Tokens
10.1K
Snippets
44
Records
49
Agent score
54%

What's inside Extralite

  1. Track and apply changes using Changesets

    main

    Note: Changeset support requires the extralite-bundle version because the SQLite session extension is disabled by default in many distributions.

    Changesets allow you to track data modifications within a block, apply those changes to other databases, or undo them.

    • Use db.track_changes(table_names) to start tracking. Pass nil to track all tables.
    • Use changeset.apply(target_db) to apply changes to a different database.
    • Use changeset.invert.apply(db) to undo the changes.
    • Use changeset.to_blob to serialize a changeset and Extralite::Changeset.new.load(blob) to restore it.
    # track changes to specific tables
    changeset = db.track_changes(:foo, :bar) do
      insert_a_bunch_of_records(db)
    end
    
    # track changes to all tables
    changeset = db.track_changes(nil) do
      insert_a_bunch_of_records(db)
    end
    
    # apply to another database
    changeset.apply(some_other_db)
    
    # undo changes
    changeset.invert.apply(db)
    
    # save and load
    IO.write('my.changes', changeset.to_blob)
    
    changeset = Extralite::Changeset.new
    changeset.load(IO.read('my.changes'))
  2. Use Structured Transforms for complex object graphs

    main

    For complex queries involving JOINs, use Extralite::Transform to map flat result rows into nested, structured Ruby objects. This allows you to use an identity map to eliminate duplicate entities (e.g., when a post has multiple tags) and represent one-to-many or many-to-many relationships naturally.

    transform = Extralite::Transform do
      {
        id:       integer.identity, # uses identity map to prevent duplicates
        content:  text,
        tags:     [{ # nested relationship
          id:     integer.identity,
          name:   text
        }]
      }
    end
    
    db.query(transform, sql)
  3. Use the Progress Handler for timeouts and instrumentation

    main

    The progress handler is a block of code executed while a query is in progress or while the database is busy. It is useful for implementing timeouts, updating UIs, or switching between fibers/threads.

    Important: You must not issue any queries from within the progress handler.

    To detect if the database is currently busy, check the first argument passed to the block. In multi-threaded apps, you should call Thread.pass inside the handler to allow other threads to run.

    # Detect busy state
    db.on_progress do |busy|
      if busy
        # handle busy state
      else
        # handle normal progress
      end
    end
    
    # Implement a timeout using the progress handler
    def setup_progress_handler
      @db.on_progress do
        raise TimeoutError if Time.now - @t0 >= @timeout
        Thread.pass
      end
    end
    
    def run_query_with_timeout(sql, timeout)
      @t0 = Time.now
      @db.query(sql)
    rescue TimeoutError
      nil
    end
  4. Thread Safety requirements for Extralite

    main

    A single Extralite::Database instance can be used in multiple threads simultaneously if and only if:

    1. No explicit transactions are used.
    2. Each thread issues queries via Database#query_xxx or uses its own Query instance.
    3. The gvl_release_threshold is not set to 0 (ensuring the GVL is released periodically).
  5. Query Modes in Extralite

    main

    Extralite provides different methods to retrieve data in various formats depending on your needs. All #query_xxx methods return an array of rows (for multiple rows), while #query_single_xxx methods return a single row/value.

    Available Modes

    • :hash (Default): Returns each row as a hash mapping column names (symbols) to values. Use #query or #query_single.
    • :array: Returns each row as an array. Use #query_array or #query_single_array.
    • :splat: Returns values without a container (unwrapped). Use #query_splat or #query_single_splat.
    # Hash mode (default)
    db.query('select 1') #=> [{ "1" => 1 }]
    
    # Array mode
    db.query_array('select 1') #=> [[1]]
    
    # Splat mode
    db.query_splat('select 1') #=> [1]
    
    # Single row variants
    db.query_single('select 1') #=> { "1" => 1 }
    db.query_single_array('select 1') #=> [1]
    db.query_single_splat('select 1') #=> 1
  6. Use Extralite with Fibers (Async/Polyphony)

    main

    In multi-fibered applications (using gems like Async or Polyphony), use the progress handler to allow the fiber scheduler to switch between fibers while a query is running.

    • For general fiber schedulers (like Async): call sleep(0) in the progress handler.
    • For Polyphony: call snooze. If the app is also multi-threaded, also call Thread.pass.
    # General fiber scheduler solution
    db.on_progress { sleep(0) }
    
    # Polyphony solution
    db.on_progress do
      snooze
      Thread.pass
    end
  7. Use the Splat Query Mode for efficiency

    main

    The :splat mode is highly efficient for retrieving single columns or when iterating with a block. It avoids the overhead of allocating arrays or hashes for every row, reducing Ruby GC pressure.

    Unwrapping single columns

    Instead of mapping an array of arrays, use query_splat to get a flat array of values:

    # Using array mode (requires manual unwrapping)
    ids = db.query_array('select id from tasks where active = 1').map { |r| r.first }
    
    # Using splat mode (direct)
    ids = db.query_splat('select id from tasks where active = 1')

    Iterating with splatted arguments

    When providing a block to #query_splat, the column values are passed as individual arguments to the block:

    db.query_splat('select a, b, c from foo') do |a, b, c|
      do_this_with(a, b)
      do_that_with(c)
    end
  8. Connect to an Extralite database using Sequel

    main

    You can use the extralite adapter to connect to an Extralite database via the Sequel ORM. Since Extralite is file-based, connection options are focused on the database file path and access modes.

    Connection Options

    • :database: The database name. This can be a filename, the special string ':memory:', or a file:// URI.
    • :readonly: If true, opens the database in read-only mode. Useful for static data.
    • :timeout: How long to wait (in milliseconds) for the database to become available if it is locked. Defaults to 5000.
    • :setup_regexp_function: If true, the adapter will register a regexp function in SQLite, allowing you to use regular expression operators in your SQL queries.

    URI Format

    To mimic the file:// URI scheme:

    • Use two preceding slashes (//path/to/db) for a relative path.
    • Use three preceding slashes (///path/to/db) for an absolute path.
    require 'sequel'
    require 'extralite'
    
    # Connect to a file-based database
    DB = Sequel.connect(:extralite, database: 'path/to/my_database.db')
    
    # Connect to an in-memory database
    DB = Sequel.connect(:extralite, database: ':memory:')
    
    # Connect with regexp support enabled
    DB = Sequel.connect(:extralite, database: 'my_db.db', setup_regexp_function: true)
  9. Install the Extralite-SQLite Bundle

    main

    If you do not have the sqlite3 library installed on your system, want to avoid using the system-installed version, or want access to the latest SQLite features (e.g., version 3.53.3), install the extralite-bundle gem. This gem bundles the SQLite source code directly.

    Usage is identical to the standard extralite gem; simply use require 'extralite' to load it.

  10. Use Prepared Queries for performance and parameterization

    main

    Prepared queries (prepared statements) maximize performance for repeated queries and allow for parameterized SQL to prevent injection. Use #prepare to create a query object, then #bind to provide parameters.

    # Create a parameterized query
    query = db.prepare('select * from foo where x = ?')
    
    # Bind and execute
    query.bind(1).to_a
    
    # Or bind during preparation
    query = db.prepare('select * from foo where x = ?', 1)
  11. Use Extralite with the Sequel ORM

    main

    Extralite provides a Sequel adapter. To use it, ensure extralite is in your Gemfile and use the extralite:// URI scheme when connecting.

    require 'sequel'
    require 'extralite'
    
    DB = Sequel.connect('extralite://blog.db')
    articles = DB[:articles]
    p articles.to_a