zombiezen/go-sqlite

repository·main·Indexed 21 days ago

https://github.com/zombiezen/go-sqlite

A low-level Go interface to SQLite 3 that provides a CGo-free implementation via modernc.org/sqlite. It is designed as a drop-in replacement for crawshaw.io/sqlite to simplify cross-compilation. The library includes the zombiezen-sqlite-migrate tool for source code migration, a Backup API for online database snapshots, and support for custom SQLite authorizers.

Tokens
18.4K
Snippets
65
Records
78
Agent score
75%

What's inside zombiezen-go-sqlite

  1. Understand automatically fixed migration changes

    main

    The zombiezen-sqlite-migrate tool performs several mechanical transformations to preserve semantics while updating the API:

    • ErrorCode to ResultCode: crawshaw.io/sqlite.ErrorCode is renamed to zombiezen.com/go/sqlite.ResultCode to better reflect that these represent SQLite result codes.
    • Constant Renaming: Constants are converted from upper snake case with SQLITE_ prefixes (e.g., sqlite.SQLITE_OK) to upper camel case with type prefixes (e.g., sqlite.ResultOK).
    • Package Relocation: sqlitex.File and sqlitex.Buffer are moved to zombiezen.com/go/sqlite/sqlitefile.
    • Session API: Various symbols in the session API are renamed for clarity.
    • FS Method Renaming: sqlitex.ExecFS, sqlitex.ExecTransientFS, and sqlitex.ExecScriptFS are renamed to sqlitex.ExecuteFS, sqlitex.ExecuteTransientFS, and sqlitex.ExecuteScriptFS respectively.
  2. Migrate from crawshaw.io/sqlite to zombiezen.com/go/sqlite

    main
    This package is designed as a mostly drop-in replacement for crawshaw.io/sqlite. It includes a go fix-like tool to assist in migrating existing codebases. For detailed instructions, refer to the migration documentation.
  3. Get started with zombiezen.com/go/sqlite

    main

    To use the library, open a connection using sqlite.OpenConn and execute queries. The package provides sqlitex for utilities like executing transient queries. This example demonstrates opening an in-memory database and executing a simple SELECT statement.

    import (
      "fmt"
    
      "zombiezen.com/go/sqlite"
      "zombiezen.com/go/sqlite/sqlitex"
    )
    
    // ...
    
    // Open an in-memory database.
    conn, err := sqlite.OpenConn(":memory:", sqlite.OpenReadWrite)
    if err != nil {
      return err
    }
    defer conn.Close()
    
    // Execute a query.
    err = sqlitex.ExecuteTransient(conn, "SELECT 'hello, world';", &sqlitex.ExecOptions{
      ResultFunc: func(stmt *sqlite.Stmt) error {
        fmt.Println(stmt.ColumnText(0))
        return nil
      },
    })
    if err != nil {
      return err
    }
  4. Use zombiezen-sqlite-migrate to migrate source code

    main

    The migration tool works in two steps: first, preview the changes to ensure they are correct, then apply them using the -w flag.

    1. Preview changes: Run the tool on your package path (e.g., ./...) to see what would be modified.
    2. Apply changes: Run the tool with the -w flag to write the changes to your files.
    # Preview changes
    zombiezen-sqlite-migrate ./...
    
    # Apply changes
    zombiezen-sqlite-migrate -w ./...
  5. Install zombiezen.com/go/sqlite

    main

    Install the package using go get. Note that while this library is CGo-free and allows building with CGO_ENABLED=0, you must ensure you are building for a supported architecture as defined by the underlying modernc.org/sqlite implementation.

    go get zombiezen.com/go/sqlite
  6. Configure SQLite connection flags with OpenFlags

    main

    The OpenFlags type is used to configure how a connection is opened via OpenConn. You must provide exactly one of the required flags (OpenReadOnly or OpenReadWrite). Other optional flags can be combined using bitwise OR (|) to modify connection behavior, such as enabling WAL mode, creating files if they don't exist, or using in-memory storage.

    // Example: Opening a database for reading and writing, creating it if it doesn't exist, and enabling WAL mode
    flags := sqlite.OpenReadWrite | sqlite.OpenCreate | sqlite.OpenWAL
    conn, err := sqlite.OpenConn("path/to/db", flags)
  7. The Blob type

    main
    The Blob type provides streaming, incremental access to SQLite BLOB columns. It implements several standard Go interfaces, including io.Reader, io.Writer, io.Seeker, io.ReaderFrom, and io.WriterTo, making it compatible with most Go standard library functions that operate on streams.
  8. Inspect SQL actions with the Action type

    main

    The Action struct provides detailed context about the specific SQL operation being authorized. You can use various methods to extract metadata depending on the OpType of the action:

    • Type(): Returns the OpType (e.g., OpRead, OpInsert, OpDropTable).
    • Database(): Returns the database name (e.g., main, temp).
    • Table(): Returns the name of the table involved.
    • Column(): Returns the column name (relevant for OpRead and OpUpdate).
    • Index(): Returns the index name.
    • View(): Returns the view name.
    • Trigger(): Returns the name of the trigger or the accessor (the inner-most trigger/view responsible for the access).
    • Pragma() / PragmaArg(): Returns details for OpPragma actions.
    • Operation(): Returns transaction/savepoint keywords (BEGIN, COMMIT, etc.).
    • File(): Returns the filename for OpAttach.
    • Module(): Returns the module name for virtual tables.
    • Savepoint(): Returns the savepoint name.
    // Example of inspecting an action
    auth := sqlite.AuthorizeFunc(func(action sqlite.Action) sqlite.AuthResult {
        if action.Type() == sqlite.OpRead {
            fmt.Printf("Reading from table: %s, column: %s\n", action.Table(), action.Column())
        }
        return sqlite.AuthResultOK
    })
  9. Understand BestIndex inputs and outputs

    main

    The BestIndex method is used by SQLite to optimize queries.

    Inputs (IndexInputs):

    • Constraints: A list of IndexConstraint objects representing the WHERE clause.
    • OrderBy: A list of IndexOrderBy objects representing the ORDER BY clause.
    • ColumnsUsed: A bitmask of columns used by the statement.

    Outputs (IndexOutputs):

    • ConstraintUsage: Maps IndexInputs.Constraints to VTableCursor.Filter arguments via ArgvIndex.
    • ID: An IndexID (number or string) returned to Filter to identify the chosen strategy.
    • OrderByConsumed: true if the output is already sorted.
    • EstimatedCost: A float6.a representing the cost (e.g., $N$ for linear scan, $\log(N)$ for binary search).
    • EstimatedRows: An estimate of the number of rows returned.
    type IndexInputs struct {
    	Constraints []IndexConstraint
    	OrderBy    []IndexOrderBy
    	ColumnsUsed uint64
    }
    
    type IndexOutputs struct {
    	ConstraintUsage  []IndexConstraintUsage
    	ID               IndexID
    	OrderByConsumed  bool
    	EstimatedCost    float64
    	EstimatedRows    int64
    	UseZeroEstimates bool
    	IndexFlags       IndexFlags
    }
  10. Resolve changeset conflicts

    main

    When applying a changeset, conflicts can occur. You resolve these by providing a ConflictHandler to ApplyChangeset.

    Conflict Types (ConflictType):

    • ChangesetData: Data in the changeset conflicts with data in the database.
    • ChangesetNotFound: The row expected by the changeset was not found.
    • ChangesetConflict: A conflict that is not covered by other types.
    • ChangesetConstraint: A constraint violation occurred.
    • ChangesetForeignKey: A foreign key constraint violation occurred.

    Conflict Actions (ConflictAction):

    • ChangesetOmit: The change that caused the conflict is ignored, and the session continues to the next change.
    • ChangesetAbort: All changes applied so far are rolled back, and ApplyChangeset returns an error.
    • ChangesetReplace:
      • If type is ChangesetData: The conflicting row is updated or deleted.
      • If type is ChangesetConflict: The conflicting row is removed, and a second attempt to apply the change is made. If the second attempt fails, the original row is restored.
      • For other types: ApplyChangeset rolls back all changes and returns a ResultMisuse error.
  11. Perform online database backups with the Backup API

    main

    The Backup API allows for copying data between two SQLite database connections (online backup). This is useful for creating snapshots of a live database without stopping writes.

    Workflow

    1. Initialize: Use NewBackup to create a backup object. You must specify the destination connection (dst), the source connection (src), and their respective database names (e.g., "main", "temp", or an attached database name).
    2. Execute: Call Step(n) repeatedly to copy pages.
      • If n is positive, it copies up to n pages.
      • If n is negative, it copies all remaining pages.
      • Step returns more bool. If more is true, there are still pages to copy.
      • If Step returns an error with ResultBusy or ResultLocked, it is a temporary error and you should retry the operation.
    3. Monitor: Use Remaining() to see how many pages are left and PageCount() to see the total pages in the source.
    4. Cleanup: You must call Close() to release resources. If the backup is not finished, Close() will roll back any active write transaction on the destination database.
    // Example: Copying 'main' from src to 'main' in dst
    backup, err := sqlite.NewBackup(dst, "main", src, "main")
    if err != nil {
    	log.Fatal(err)
    }
    defer backup.Close()
    
    for {
    	more, err := backup.Step(-1) // Copy all remaining pages
    	if err != nil {
    		// Handle retriable errors like SQLITE_BUSY
    		break
    	}
    	if !more {
    		break
    	}
    }