sqinn-go

repository·master·Indexed 19 days ago

https://github.com/cvilsmeier/sqinn-go

A Go library for accessing SQLite databases without using cgo. It operates by running the sqinn program as a child process and communicating via stdin/stdout/stderr, facilitating easy cross-compilation and removing the need for a local GCC installation. The library provides high-level interfaces for executing SQL commands and querying rows, and includes embedded prebuilt binaries for linux_amd64 and windows_amd64.

Tokens
3.1K
Snippets
10
Records
13
Agent score
68%

What's inside sqinn-go

  1. How concurrency works in Sqinn-Go

    master

    A single sqinn instance is inherently single-threaded; requests are served one after another.

    To achieve true database-level concurrency, you should:

    1. Spin up multiple sqinn instances.
    2. Implement your own connection pooling.

    Warning: When accessing a SQLite database concurrently via multiple instances, you may encounter SQLITE_BUSY errors. Using PRAGMA busy_timeout in your SQL commands can help mitigate this.

  2. When to build sqinn manually instead of using prebuilt binaries

    master

    While sqinn-go provides prebuilt binaries for common platforms, you should build sqinn yourself in the following scenarios:

    1. Security/Trust: If you do not trust the binaries provided by GitHub runners.
    2. Unsupported Platforms: If you require sqinn for a platform not officially supported by the embedded set (e.g., arm32).
    3. Special SQLite Features: If you require specific SQLite configurations that are not present in the default builds, such as an mmap size greater than 2G.
  3. Basic usage of Sqinn-Go

    master

    Sqinn-Go allows you to execute SQL commands and query rows without using cgo. It works by launching a sqinn child process and communicating via stdin/stdout.

    Key workflow:

    1. Launch an instance using sqinn.MustLaunch with sqinn.Options.
    2. Execute commands using MustExecSql or MustExecParams.
    3. Query data using MustQueryRows, specifying the types of columns you wish to fetch.
    4. Close the instance with Close().

    Note: Sqinn-Go is not a database/sql driver. It uses higher-level interfaces to minimize the inter-process communication (IPC) overhead.

    import (
    	"fmt"
    	"github.com/cvilsmeier/sqinn-go/v2"
    )
    
    func main() {
    	// Launch sqinn, close when done.
    	sq := sqinn.MustLaunch(sqinn.Options{
    		Db: ":memory:", // use a transient in-memory database
    	})
    	defer sq.Close()
    
    	// Create a table, cleanup when done
    	sq.MustExecSql("CREATE TABLE users (id INTEGER PRIMARY KEY NOT NULL, name TEXT)")
    	defer sq.MustExecSql("DROP TABLE users")
    
    	// Insert users
    	sq.MustExecParams("INSERT INTO users (id, name) VALUES (?, ?)", 3, 2, []sqinn.Value{
    		sqinn.Int32Value(1), sqinn.StringValue("Alice"),
    		sqinn.Int32Value(2), sqinn.StringValue("Bob"),
    		sqinn.Int32Value(3), sqinn.StringValue("Carol"),
    	})
    
    	// Query users
    	rows := sq.MustQueryRows(
    		"SELECT id, name FROM users WHERE id >= ? ORDER BY id",
    		[]sqinn.Value{sqinn.Int32Value(0)},      // query parameters
    		[]byte{sqinn.ValInt32, sqinn.ValString}, // fetch id as int, name as string
    	)
    	for _, values := range rows {
    		fmt.Printf("user id=%d, name=%s\n", values[0].Int32, values[1].String)
    	}
    }
  4. Run Sqinn-Go tests and check coverage

    master

    To execute the automated unit tests on linux/amd64 or windows/amd64, follow these steps:

    1. Initialize a test module and install the library:
    go mod init test
    go get -v -u github.com/cvilsmeier/sqinn-go/v2
    1. Run the tests:
    go test github.com/cvilsmeier/sqinn-go/v2
    1. Check test coverage:
    go test github.com/cvilsmeier/sqinn-go/v2 -coverprofile=cover.out
    go tool cover -html=cover.out
    go mod init test
    go get -v -u github.com/cvilsmeier/sqinn-go/v2
    go test github.com/cvilsmeier/sqinn-go/v2
  5. Update embedded sqinn binaries in sqinn-go

    master

    Sqinn-go embeds prebuilt sqinn binaries for specific platforms to provide convenience. If you need to update these embedded binaries (for example, to use a newer version or a custom build), you must download the latest sqinn releases, unzip them, and then compress the binaries into .gz files within the src/prebuilt directory of the sqinn-go repository.

    Supported platforms currently embedded are:

    • linux_amd64
    • windows_amd64
    • darwin_amd64
    • darwin_arm64
    # 1. Unzip the downloaded releases into platform-specific directories
    cd ~/Downloads
    unzip dist-darwin-amd64.zip   -d dist-darwin-amd64
    unzip dist-darwin-arm64.zip   -d dist-darwin-arm64
    unzip dist-linux-amd64.zip    -d dist-linux-amd64
    unzip dist-windows-amd64.zip  -d dist-windows-amd64
    
    # 2. Compress the binaries into the sqinn-go prebuilt directory
    cd /path/to/sqinn-go/src/prebuilt
    cat ~/Downloads/dist-linux-amd64/sqinn       | gzip > linux-amd64.gz
    cat ~/Downloads/dist-windows-amd64/sqinn.exe | gzip > windows-amd64.gz
    cat ~/Downloads/dist-darwin-amd64/sqinn      | gzip > darwin-amd64.gz
    cat ~/Downloads/dist-darwin-arm64/sqinn      | gzip > darwin-arm64.gz
  6. Configure a custom Sqinn binary path

    master

    By default, the library uses embedded prebuilt binaries for linux_amd64 and windows_amd64. If you have compiled your own sqinn binary, you can specify its location in the sqinn.Options struct using the Sqinn field.

    sq := sqinn.MustLaunch(sqinn.Options{
    	Sqinn: "/path/to/sqinn",
    })
  7. Configure sqinn.Options

    master

    The sqinn.Options struct defines how the sqinn subprocess is initialized:

    FieldTypeDescription
    SqinnstringPath to sqinn executable. Use sqinn.Prebuilt (":prebuilt:") to use the embedded binary (supports linux/amd64 and windows/amd64).
    DbstringDatabase filename or path (e.g., "/tmp/test.db" or ":memory:").
    LoglevelintLogging level: 0 (off), 1 (info), or 2 (debug).
    LogfilestringFilename where sqinn will print log messages.
    Logfunc(string)A callback function to handle log messages from the sqinn process.

    Note: The default Sqinn is :prebuilt: and the default Db is :memory:.

  8. Query SQL results with Query()

    master

    The Query method executes a SQL statement and fetches result rows using a callback.

    Parameters:

    • sql: The SQL query string.
    • params: A slice of sqinn.Value for parameter binding.
    • coltypes: A byte slice defining the expected types for each column (e.g., using ValInt32, ValString, etc.).
    • consume: A callback func(row int, values []Value) called for every row returned.

    Alternatively, use QueryRows(sql string, params []Value, coltypes []byte) to fetch all rows into a [][]Value slice at once.

    // Example: Querying rows and consuming them via callback
    coltypes := []byte{sqinn.ValInt32, sqinn.ValString}
    err := s.Query("SELECT id, name FROM users", nil, coltypes, func(row int, values []sqinn.Value) {
        id, _ := sqinn.Scan(values).NextInt32()
        name, _ := sqinn.Scan(values).NextString()
        fmt.Printf("Row %d: %d - %s\n", row, id, name)
    })
  9. Execute SQL statements with Exec()

    master

    The Exec method runs a SQL statement. It supports multiple iterations and parameter binding via a ProduceFunc callback.

    • niterations: How many times to execute the SQL. If 0, it is a no-op. If 1, it runs once.
    • nparams: How many parameters to bind per iteration.
    • produce: A callback func(iteration int, params []Value) used to populate the params slice for each iteration.

    For simple single-statement execution without parameters, use ExecSql(sql string).

    // Example: Executing a statement with parameters across 3 iterations
    err := s.Exec("INSERT INTO users (name) VALUES (?)", 3, 1, func(iteration int, params []sqinn.Value) {
        params[0] = sqinn.StringValue(fmt.Sprintf("user_%d", iteration))
    })
  10. Launch a sqinn instance

    master

    Use sqinn.Launch(opt Options) to start a new sqinn subprocess. This provides an interface to SQLite databases without using cgo. You can specify the path to the sqinn executable, the database file (or :memory:), and logging preferences.

    If you use the special name :prebuilt:, the library will automatically extract an embedded sqinn binary for linux/amd64 or windows/amd64 into a temporary directory and execute it.

    import "github.com/cvilsmeier/sqinn-go/v2"
    
    opt := sqinn.Options{
        Sqinn: sqinn.Prebuilt,
        Db:    "/path/to/database.db",
    }
    
    s, err := sqinn.Launch(opt)
    if err != nil {
        panic(err)
    }
    defer s.Close()
  11. Use sqinn.Value and Bind() for data handling

    master

    The sqinn.Value type is a container for SQLite data types. Supported types are:

    • ValNull (0)
    • ValInt32 (1)
    • ValInt64 (2)
    • ValDouble (3)
    • ValString (4)
    • ValBlob (5)

    Use the helper functions to create values:

    • NullValue()
    • Int32Value(int)
    • Int64Value(int64)
    • DoubleValue(float64)
    • StringValue(string)
    • BlobValue([]byte)

    To convert a slice of standard Go types (nil, int, int64, float64, string, []byte) into []sqinn.Value, use sqinn.Bind(params []any).

    // Using Bind to prepare parameters
    params := sqinn.Bind([]any{123, "hello", nil})
    // params is now []sqinn.Value{Int32Value(123), StringValue("hello"), NullValue()}