DOCSEARCHbyAlgolia
Submit Docs

Turso SQL Database Engine

repository·main·Indexed 12 days ago

https://github.com/tursodatabase/turso

An in-process SQL database engine written in Rust, designed for high compatibility with SQLite and support for multiple SQL dialects via a VDBE-based architecture. It includes a Rust-based implementation of the SQLite3 C API (compatible with version 3.42.0), a .NET ADO.NET provider (Turso.Data.Sqlite.Provider) supporting NativeAOT static linking, and a managed HTTP client (Turso.Platform.Client) for the Turso Platform API.

Tokens
299.3K
Snippets
997
Records
1.3K
Agent score
92%

What's inside Turso

  1. Overview of the Join Order Benchmark (JOB)

    main

    The Join Order Benchmark (JOB) is a suite of 113 analytical queries based on the IMDB schema. It is designed to stress-test query optimizer join-order planning using realistic correlated data.

    In this repository, the benchmark is used to track statement preparation costs (parsing, planning, and code generation) via core/benches/prepare_benchmark.rs. Note that for these specific preparation benchmarks, actual IMDB data is not required.

  2. Overview of @tursodatabase/database

    main

    The @tursodatabase/database package is an embedded SQLite-compatible database library designed for Node.js. It runs in-process, meaning there is no network overhead.

    Key Features:

    • SQLite Compatible: Supports SQLite query language and file formats.
    • In-process: Runs directly within your Node.js process.
    • TypeScript Support: Includes full type definitions.
    • Cross-platform: Supports Linux (x86/arm64), macOS, and Windows.

    Note: For browser support, use the @tursodatabase/database-wasm package instead.

  3. Overview of the TPC-DS query set

    main

    The TPC-DS query set is a collection of 99 benchmark queries and a standard schema consisting of 24 tables. It is used to measure statement preparation costs (parse, plan, and codegen) via core/benches/prepare_benchmark.rs.

    Note that this specific set is designed for tracking preparation costs, so actual data population is not required for these benchmarks.

    Important Compatibility Note: Certain SQL features are currently unsupported by Turso and are skipped during benchmarks. If you are testing new features, ensure they are removed from the internal skip list in the benchmark runner. Unsupported features include:

    • ROLLUP
    • stddev_samp
    • Custom window frames
    • Parenthesized compound selects
    • Non-equality FULL OUTER JOIN

  4. Overview of Turso Database

    main
    Turso is an in-process SQL database written in Rust that is compatible with SQLite. It functions as a virtual machine that compiles SQL into bytecode for a custom Virtual Database Engine (VDBE). This architecture allows Turso to support multiple SQL dialects by using different frontends (such as SQLite and Postgres) that compile down to the same core engine. It is designed to be a reliable, modern core for various database frontends, similar to how LLVM serves compilers.

  5. Overview of the Turso SDK kit

    main
    The Turso SDK kit provides a low-level API designed to simplify the development of language-specific SDKs. The core logic is implemented in the Rust layer and is propagated to a C API. This architecture allows for easy consumption of bindings across various programming languages. A key feature of Turso is its support for async IO execution, which can leverage modern storage backends like io_uring.

  6. Overview of @tursodatabase/database features

    main

    The @tursodatabase/database package provides an in-process, SQLite-compatible database engine for JavaScript/TypeScript environments.

    Key features include:

    • SQLite Compatibility: Supports SQLite query language and file formats.
    • In-process Execution: Runs directly in the Node.js process with no network overhead.
    • Cross-platform: Supports Linux (x86/arm64), macOS, Windows, and Browsers (via WASM).
    • TypeScript Support: Includes full type definitions.

  7. Overview of the Turso extension API

    main

    The turso_ext crate provides an ergonomic Rust API for creating and registering libraries that extend Turso functionality. These extensions can be loaded similarly to traditional sqlite3 extensions.

    Currently supported extension types include:

    • Scalar Functions: Created using the scalar macro.
    • Aggregate Functions: Defined using the AggregateDerive macro and the AggFunc trait.
    • Virtual Tables: Created by implementing a module with the VTabModuleDerive macro and the VTabCursor trait.
    • VFS Modules: Used to extend Turso's OS interface by implementing the VfsExtension and VfsFile traits.

  8. Overview of Turso SQL Reference structure

    main

    The Turso SQL reference is organized into several key areas covering language syntax, Turso-specific extensions, and CLI usage:

    • Core SQL: Includes data-types.mdx (storage classes, type affinity, STRICT tables), expressions.mdx (literals, operators, CAST, CASE), and statements/ (DDL/DML like select.mdx, insert.mdx, and transactions.mdx which includes Turso MVCC BEGIN CONCURRENT).
    • Functions: Covers scalar.mdx, aggregate.mdx, json.mdx, and Turso-specific features like vector.mdx (vector search) and fts.mdx (full-text search).
    • Turso Extensions & Config: Includes pragmas.mdx (CDC, encryption), extensions.mdx (UUID, regexp, vector, time, CSV, percentile), and compatibility.mdx (SQLite compatibility notes).
    • CLI Reference: Contains getting-started.mdx (installation, output modes), command-line-options.mdx (flags and arguments), and shell-commands.mdx (dot commands like .tables, .schema).
    • Experimental Features: Instructions for enabling experimental features across the CLI and all SDKs are located in experimental-features.mdx.

  9. Overview of @tursodatabase/database-wasm

    main

    The @tursodatabase/database-wasm package is an embedded SQLite-compatible database library designed for JavaScript in the browser.

    Key Features:

    • SQLite Compatibility: Supports SQLite query language and file formats.
    • In-process: Runs directly in the environment with no network overhead.
    • TypeScript Support: Includes full type definitions.

    Related Packages:

    • @tursodatabase/serverless: Provides a serverless driver with the same API.
    • @tursodatabase/sync: Provides bidirectional sync between a local Turso database and Turso Cloud.

  10. Javascript Turso usage examples

    main

    The examples/ directory contains several implementation patterns for using Turso in Javascript environments. Depending on your target runtime (Node.js vs. Browser) and your data synchronization requirements (Local-only vs. Bidirectional Sync with Turso Cloud), you can refer to the following examples:

    Node.js Examples

    • Local File Database (No Sync): Use database-node for standard Node.js applications interacting with a local file-based database.
    • Bidirectional Sync: Use sync-node to implement bidirectional synchronization between a Node.js environment and Turso Cloud.

    Browser (WASM) Examples

    • Local Browser Database: Use database-wasm-vite to run a local database directly in the browser using WebAssembly (WASM).
    • Bidirectional Sync: Use sync-wasm-vite to run a WASM-based database in the browser that synchronizes bidirectionally with Turso Cloud.

    Advanced Patterns

    • Concurrent Writes: The concurrent-writes example demonstrates how to handle Multi-Version Concurrency Control (MVCC) by using BEGIN CONCURRENT to support up to 16 concurrent writers.

  11. Understand the Turso SQLite3 C API Implementation

    main
    The bindings/c directory provides a Rust-based implementation of the SQLite3 C API. This acts as a compatibility layer that allows existing applications written for SQLite to interact with Turso's native Rust database implementation without code modifications. It maps SQLite operations to Turso operations while maintaining API compatibility with SQLite version 3.42.0.

  12. Features and Roadmap of Turso

    main

    Turso provides several core features and experimental capabilities:

    Core Features

    • SQLite Compatibility: Supports SQLite SQL dialect, file formats, and the C API.
    • BEGIN CONCURRENT: Enables improved write throughput using multi-version concurrency control (MVCC).
    • Change Data Capture (CDC): Real-time tracking of database changes.
    • Multi-language Support: Bindings available for Go, JavaScript, Java, .NET, Python, Rust, and WebAssembly.
    • Asynchronous I/O: Support for io_uring on Linux.
    • Cross-platform: Runs on Linux, macOS, Windows, and in browsers via WebAssembly.
    • Vector Support: Includes exact search and vector manipulation.
    • Improved Schema Management: Extended ALTER support and faster schema changes.

    Experimental Features

    • Postgres Compatibility: Supports Postgres SQL dialect and wire protocol.
    • Encryption at Rest: Local data protection.
    • Incremental Computation: Uses DBSP for incremental view maintenance and query subscriptions.
    • Full-Text-Search: Powered by the tantivy library.
    • Multi-process WAL Coordination: Uses a .tshm sidecar for cross-process WAL readers and writers.

    Roadmap

    • Vector Indexing: Fast approximate vector search (similar to libSQL vector search).