DOCSEARCHbyAlgolia
Submit Docs

CozoDB Documentation

repository·main·Indexed 26 days ago

https://github.com/cozodb/cozo

A high-performance, embeddable, relational database using Datalog for querying. Optimized for graph data, CozoDB supports complex recursive queries, time-travel capabilities, and vector search via HNSW indices. It provides a standalone executable, a REPL, an HTTP API, and native bindings for C, Java, Clojure, Android, Node.js, Python, Swift, and WASM.

Tokens
22.9K
Snippets
44
Records
153
Agent score
86%

What's inside CozoDB

  1. Overview of CozoDB

    main

    CozoDB is a general-purpose, transactional, relational database that uses Datalog for querying. It is designed to be embeddable (running in the same process as your application) while also supporting client-server mode for high concurrency and large datasets.

    Key features include:

    • Graph Focus: Uses a relational model to handle interconnected graph data and algorithms.
    • Datalog Querying: Supports powerful, composable queries and efficient recursion (including canned algorithms like PageRank).
    • Time Travel: Allows tracking changes over time and querying historical views of data (enabled per-relation to manage overhead).
    • High Performance: Optimized for both OLTP (high QPS) and OLAP (fast scans) workloads, with support for various storage engines like RocksDB.

  3. Vector Search and HNSW Indices in CozoDB

    main

    As of version v0.6, CozoDB supports vector search using HNSW (hierarchical navigable small world) indices integrated directly within Datalog.

    Capabilities include:

    • HNSW Indices: Create indices on relations containing vectors. You can specify filters to dictate which rows or specific vectors within a row are indexed.
    • Datalog Integration: Vector search is treated as a unification process, allowing you to use vectors (explicitly provided or retrieved from other relations) as pivots for joins.
    • Recursive Vector Search: Because it is integrated into Datalog, vector search can be used within recursive queries.
    • Graph Exposure: HNSW indices are exposed as proximity graphs, allowing users to apply classical graph algorithms like community detection directly to them.
    • Concurrency & Memory: Uses Multi-Version Concurrency Control (MVCC) to protect indices during concurrent writes and is optimized for low memory usage via Rust's RAII.

  5. Understand Cozo Architecture

    main

    Cozo's architecture consists of three main layers:

    1. Language/Environment Wrappers: These wrap the Rust API for different runtimes. For example, the standalone cozo service wraps the Rust API as HTTP endpoints, while cozo-node wraps synchronous Rust APIs into asynchronous JavaScript APIs.
    2. Query Engine: The core of the project. It handles function implementations, aggregation operators, schema definitions, database transactions, query compilation, and query execution.
    3. Storage Engine: Defines a storage interface (a Rust trait) for key-value storage and range scanning of binary data. Supported implementations include Memory (M), SQLite (Q), RocksDB (R), Sled (S), and TiKV (T).

    Data Format Note: Cozo uses a row-oriented binary storage format. It uses the memcomparable method to store composite keys as byte arrays, ensuring correct semantic sorting via byte-order sorting. Because of this, querying SQLite-stored data directly with standard SQL may result in unreadable output.

  6. Install CozoDB for various environments

    main

    CozoDB is highly embeddable and supports many languages and platforms. Installation depends on your target environment. Supported storage backends include:

    • M: In-memory, non-persistent
    • Q: SQLite
    • R: RocksDB
    • S: Sled
    • T: TiKV (distributed)
    Language/EnvironmentOfficial Platform SupportStorage
    PythonLinux (x86_64), Mac (ARM64, x86_64), Windows (x86_64)MQR
    NodeJSLinux (x86_64, ARM64), Mac (ARM64, x86_64), Windows (x86_64)MQR
    Web browserModern browsers supporting WebAssemblyM
    Java (JVM)Linux (x86_64, ARM64), Mac (ARM64, x86_64), Windows (x86_64)MQR
    Clojure (JVM)Linux (x86_64, ARM64), Mac (ARM64, x86_64), Windows (x86_64)MQR
    AndroidAndroid (ARM64, ARMv7, x86, x86_64)MQ
    iOS/MacOS (Swift)iOS (ARM64, simulators), Mac (ARM64, x86_64)MQ
    RustSource only, any platform with std supportMQRST
    GolangLinux (x86_64, ARM64), Mac (ARM64, x86_64), Windows (x86_64)MQR
    C/C++/C FFILinux (x86_64, ARM64), Mac (ARM64, x86_64), Windows (x86_64)MQR
    Standalone HTTP serverLinux (x86_64, ARM64), Mac (ARM64, x86_64), Windows (x86_64)MQRS T
    LispLinux (x86_64)MR
    SmalltalkWin10 & Linux (Ubuntu 23.04) x86_64MQR

  7. Build Cozo-lib-java for JDK

    main

    If prebuilt binaries are insufficient, you can build the library for the JDK using the Rust toolchain. To include RocksDB storage support, use the -F storage-rocksdb feature flag.

    cargo build --release -p cozo_java -F storage-rocksdb

  9. Compile Cozo WASM module

    main

    To compile the Cozo WASM module yourself, you need Rust, NodeJS (with npm), and wasm-pack. To ensure compatibility with the standard browser usage instructions, you must build using the --target web flag.

    wasm-pack build --target web --release

  10. Build the cozo_embedded Python library from source

    main

    To compile the cozo_embedded library, you must have the Rust toolchain and maturin installed. You can build the library with specific features enabled for compact storage and RocksDB support using the following command:

    maturin build -F compact -F storage-rocksdb --release

  11. Compile Cozo Java for JDK

    main

    To compile the Cozo Java library for the JDK from source, ensure you have the Rust toolchain installed, then run the following command. This example includes the storage-rocksdb feature:

    cargo build --release -p cozo_java -F storage-rocksdb

  12. Build Cozo Swift from source

    main

    To build the library from source, ensure the Rust toolchain is installed, then run the build-rust.sh script.

    Optimization: Setting the environment variable CARGO_PROFILE_RELEASE_LTO=fat will result in faster libraries at the cost of longer compilation times.

    Enabling RocksDB: By default, pre-compiled binaries only support the storage-sqlite engine. To include RocksDB, modify the cargo build commands in the build script to include the -F storage-rocksdb feature flag:

    cargo build -p cozo-swift -F compact -F storage-rocksdb --target x86_64-apple-darwin --release
cargo build -p cozo-swift -F compact -F storage-rocksdb --target aarch64-apple-darwin --release

    Xcode Configuration: When using the generated library, you must link to the libc++ dynamic library in Xcode.