DOCSEARCHbyAlgolia
Submit Docs

Fluree DB Documentation

repository·main·Indexed 19 days ago

https://github.com/fluree/db

A temporal, verifiable graph database designed for high performance and AI integration. Fluree DB supports SPARQL, JSON-LD, and openCypher, featuring time travel, git-like branching, and integrated vector and full-text search. It provides a RESTful HTTP API, a Model Context Protocol (MCP) server for AI agents, and specialized tools like Agent JSON for token-efficient LLM query output and Fluree Memory for coding assistants.

Tokens
639.6K
Snippets
1.5K
Records
2.5K
Agent score
59%

What's inside Fluree DB

  1. Overview of Fluree Security and Policy

    main

    Fluree provides a multi-layered security model designed to protect data at rest and in transit, while offering fine-grained access control. Key components include:

    • Authentication: Supports decentralized identities (DIDs), signed requests, and Bearer tokens via multiple modes (did:key, standalone server tokens, OIDC/OAuth2).
    • Data Encryption: Transparent AES-256-GCM encryption for data at rest, configurable via environment variables.
    • Commit Integrity: Uses Ed25519 signatures to provide cryptographic proof of which node authored a commit.
    • Policy System: A data-centric access control model where policies are stored as RDF triples, allowing for versioning, time-travel, and auditability.

  2. Overview of Fluree DB capabilities

    main

    Fluree DB is a high-performance graph database with the following core capabilities:

    • Query Languages: SPARQL 1.1, JSON-LD Query, and openCypher.
    • Data Formats: JSON-LD, Turtle, TriG, N-Triples, and N-Quads.
    • Search: Integrated BM25 full-text search and Vector search (embedded HNSW or remote service).
    • Time Travel: Query data using transaction numbers, ISO timestamps, or commit IDs.
    • Reasoning: Support for RDFS, OWL 2 QL, OWL 2 RL, and Datalog rules.
    • Storage: Supports Memory, file, AWS S3 + DynamoDB, and IPFS.
    • Advanced Graph Features: Property-graph edges, statement-level metadata (RDF 1.2 / SPARQL 1.2), and SHACL validation.
    • Security & Verifiability: Triple-level policy enforcement and JWS-signed transactions/Verifiable Credentials.
    • Operations: Branching (forking ledgers), replication (push/pull), and OpenTelemetry observability.

  3. Overview of Fluree Indexing and Search

    main

    Fluree provides several indexing and search capabilities that extend beyond standard graph queries. These include:

    • Core Indexes: Automatic indexes (SPOT, POST, OPST, PSOT) used for query performance.
    • Inline Fulltext Search: BM25-ranked text scoring for small-to-medium corpora (up to hundreds of thousands of documents) with zero configuration.
    • BM25 Full-Text Search: Dedicated, large-scale indexes (1M+ documents) using the BM25 ranking algorithm and Block-Max WAND for efficient top-k queries.
    • Vector Search: Approximate nearest neighbor (ANN) search for embeddings using HNSW indexes.
    • Geospatial: Support for geographic point data using the geo:wktLiteral datatype and optimized 60-bit lat/lng encoding.

  4. Getting Started with Fluree DB

    main

    Fluree is a temporal graph database that stores data as RDF triples. It provides built-in support for features like Time Travel (querying historical state), Full-Text Search (BM25 indexing), Vector Search (ANN queries), Policy Enforcement (data-level access control), and Verifiable Data (cryptographically signed transactions).

    Depending on your use case, you can interact with Fluree in two ways:

    1. Server-based (HTTP API): Run the Fluree server and interact via HTTP requests.
    2. Embedded (Rust Library): Integrate Fluree directly into your Rust applications as a dependency.

  5. Overview of Fluree Memory

    main

    Fluree Memory provides a long-term, structured "project brain" for AI coding assistants (like Claude Code, Cursor, and VS Code Copilot). It allows agents to capture and retrieve facts, decisions, and constraints as structured memories stored in a local Fluree ledger.

    Key Characteristics

    • Local-first & Git-compatible: Memories are stored as plain-text Turtle (TTL) files within your project directory (.fluree-memory/repo.ttl for team-wide knowledge and .fluree-memory/.local/user.ttl for personal knowledge). This allows you to commit, diff, and review memories using standard Git workflows.
    • Structured Knowledge: Unlike unstructured markdown files, memories use a schema including kind, tags, scope, and rationale to ensure high-quality retrieval.
    • Ranked Recall: Uses BM25 keyword scoring combined with metadata re-ranking (tags, branch affinity, recency) to provide agents with targeted context rather than overwhelming them with irrelevant data.
    • Privacy & Security: Content is scanned for credential patterns on write, and matches are automatically redacted. No data leaves your local machine or infrastructure.

  6. Overview of Fluree DB

    main

    Fluree DB is a semantic graph database designed for data that requires time travel, branching, and verifiability. It stores data as an RDF knowledge graph and is built on W3C standards.

    Key capabilities include:

    • Multi-model Querying: Supports SPARQL 1.1, JSON-LD Query, and openCypher.
    • Time Travel & Branching: Every transaction is an immutable commit, allowing you to query the graph at any point in time or create isolated branches for staging changes.
    • Verifiable Data: Supports signed transactions (JWS / W3C Verifiable Credentials) and content-addressed commit chains to detect tampering.
    • Integrated Search: Includes built-in BM25 full-text indexing and HNSW vector search.
    • Reasoning: Supports OWL/RDFS inference and Datalog rules within the query engine.
    • Flexible Storage: Supports local filesystems, S3 + DynamoDB, or IPFS.
    • Embeddable: Can be used as a standalone server over HTTP or embedded directly into Rust applications.

  7. Overview of Fluree Crate Architecture

    main

    Fluree is organized into several specialized Rust crates categorized by their functional role in the database lifecycle. Understanding this organization helps in selecting the correct dependencies for your specific use case (e.g., storage, querying, or graph processing).

    Core & Foundation

    • fluree-vocab: RDF vocabulary constants and namespaces.
    • fluree-db-core: Runtime-agnostic core types (Flake, Sid, IndexType) and index structures.
    • fluree-db-novelty: Manages in-memory uncommitted data (novelty) and commit metadata.

    Graph Processing

    • fluree-graph-ir: Format-agnostic RDF intermediate representation.
    • fluree-graph-json-ld: JSON-LD expansion, compaction, and context handling.
    • fluree-graph-turtle: Turtle (TTL) parsing.
    • fluree-graph-format: RDF formatters (JSON-LD, Turtle, etc.).

    Query & Transaction

    • fluree-db-query: JSON-LD Query engine with statistics-driven planning.
    • fluree-db-sparql: SPARQL parser and lowering to internal IR.
    • fluree-db-transact: Transaction processing (parsing JSON-LD transactions into flakes/commits).

    Storage, Connection & Indexing

    • fluree-db-connection: Storage abstraction (Memory, File, Cloud).
    • fluree-db-storage-aws: AWS-specific backends (S3, DynamoDB).
    • fluree-db-nameservice: Ledger metadata and branch management.
    • fluree-db-binary-index: Binary index wire formats and read-side runtime.
    • fluree-db-indexer: Index building and orchestration.
    • fluree-db-ledger: Combines indexed DB with novelty for complete state snapshots.

    Security, Validation & Reasoning

    • fluree-db-policy: Policy enforcement and query augmentation.
    • fluree-db-credential: JWS and VerifiableCredential verification.
    • fluree-db-crypto: AES-256-GCM storage encryption.
    • fluree-db-shacl: SHACL validation engine.
    • fluree-db-reasoner: OWL2-RL reasoning engine.

  8. Overview of the Fluree HTTP API

    main
    The Fluree HTTP API provides RESTful endpoints for all database operations, including transactions, queries, and administrative tasks. It follows REST principles using resource-oriented URLs, standard HTTP methods (GET, POST), and stateless requests. The API supports content negotiation, allowing clients to specify formats for both requests and responses.

  9. Manage BM25 full-text search indexes with the Fluree CLI

    main

    The fluree bm25 command is used to manage BM25 full-text search indexes. A BM25 index is a Fluree graph source built over a source ledger. Once created, the index is queryable via an FQL f:searchText clause or the fluree-search-httpd service.

    Execution Modes

    Every subcommand can be run in one of two ways:

    1. Against a server: Use the --remote <name> flag to target a configured remote. If no flag is provided, the CLI automatically attempts to connect to a locally-running server (e.g., started via fluree server start). This mode uses HTTP endpoints like POST /v1/fluree/bm25/create and POST /v1/fluree/bm25/sync.
      • Note on Authentication: If a local server is running in admin_auth mode, the automatic local-server route will fail with a 401 because it sends no credentials. In this case, use --direct or configure the server as a remote and use --remote.
    2. In-process against local storage: Use the --direct flag to force the command to run against local files. This is useful for running commands via docker exec against a directory a running server is already serving, as native file storage uses per-file advisory locks rather than an exclusive whole-store lock.

    --remote is specified per-subcommand, while --direct is a global flag that can be placed anywhere on the command line.

  10. Understand SPARQL 1.2 Wave 1 Audit Findings

    main
    This document tracks the burn-down of SPARQL 1.2 Wave 1 compliance, specifically focusing on Turtle-star Ingest and various mini-features. It identifies root causes for failures in areas such as codepoint escaping, syntax validation, RDF 1.1 literal identity, grouping semantics, expression evaluation (EBV), and new language-direction (base-dir) functions.

  11. Understanding SPARQL 1.2 Wave 1 Audit Scope

    main

    The SPARQL 1.2 Wave 1 audit focuses on 'Turtle-star Ingest' and several 'Mini-features'. This wave covers 66 tests across several clusters:

    • SPARQL12_VERSION (3 tests)
    • SPARQL12_LANG_BASEDIR (10 tests)
    • SPARQL12_CODEPOINT_ESCAPES (6 tests)
    • SPARQL12_RDF11 (3 tests)
    • SPARQL12_GROUPING (1 test)
    • SPARQL12_EXPRESSION (1 test)
    • SPARQL12_SYNTAX (2 tests)
    • SPARQL12_EVAL_TRIPLE_TERMS (41 tests)

    Key Distinction: Most evaluation tests require both Wave 1 (data load/ingest) and Wave 2 (query syntax, specifically << >> patterns) to pass. Only a small subset passes on ingest alone.

  12. Understand Virtual-Dataset (Iceberg / R2RML) correctness and performance findings

    main

    This document serves as a findings register for parity testing between Fluree's native engine and the Virtual-Dataset (Iceberg / R2RML) implementation. It categorizes discrepancies into correctness bugs (silent-empty, silent-divergence), corpus defects (nondeterministic selection), and performance issues (DNF - Did Not Finish).

    Key Metrics for Parity:

    • hash: EQ (identical result-hashes), NE (different hashes but results are valid), or — (not comparable due to error/DNF).
    • v_scan: Count of r2rml.scan_table spans on the virtual representation.
    • pruned/sel: Ratio of files_pruned to files_selected during the scan.

    Summary of Findings (SF01 Parity Run):

    • Correctness: 7 of 16 smoke queries are correct and hash-equal. Remaining queries suffer from silent-empty returns (F1, F2), a silent-divergence in bound-subject wildcards (F3), and performance DNFs (F5).