DOCSEARCHbyAlgolia
Submit Docs

Trustfall Query Engine

repository·main·Indexed 25 days ago

https://github.com/obi1kenobi/trustfall

A versatile query engine that uses a GraphQL-like syntax to query diverse data sources, including APIs, databases, files on disk, and AI models. It provides a Rust-based core with several crates (trustfall, trustfall_core, trustfall_derive, trustfall_wasm), Python bindings via pytrustfall, and a CLI tool called trustfall_stubgen for autogenerating Rust adapter stubs from schemas.

Tokens
28.4K
Snippets
84
Records
180
Agent score
80%

What's inside Trustfall

  1. Understand Trustfall core terminology

    main

    Trustfall uses a graph-based terminology to describe data structures and relationships. Understanding these concepts is essential for writing queries and defining schemas:

    • schema: A formal description of the shape of a dataset used for validation, type-checking, and IDE auto-complete.
    • vertex: A single data item (equivalent to a SQL row).
    • vertex type: The shape/structure of vertices (equivalent to a SQL table).
    • property: A named and typed value within a vertex (equivalent to a SQL column).
    • edge: A named relationship between vertex types or specific vertices (equivalent to a SQL foreign key).
    • entrypoint: A specific starting point for queries in a schema. Unlike SQL, Trustfall uses entrypoints to model data source restrictions (e.g., when you cannot enumerate all items in a dataset, you must start from a specific entrypoint).
    • adapter: A plugin that connects the Trustfall query interpreter to an underlying data source (API, database, file, etc.).
    • directive: A query component prefixed with @ (e.g., @filter, @output) used to describe operations like filtering or transforming data.

  2. Understand the Trustfall query engine

    main

    Trustfall is a query engine designed to treat any data source—including REST APIs, JSON files, and source code repositories—as a database that can be explored, queried, and combined. It is particularly useful for querying data that is not natively in a SQL format and for joining multiple disparate data sources together.

    Key capabilities include:

    • Querying over non-SQL sources (REST, JSON, etc.).
    • Support for recursive and left joins.
    • Aggregations and arbitrary filter clauses.
    • Lazy evaluation, which is optimized for expensive or rate-limited data retrieval (e.g., pay-per-request APIs).

  3. Understand Trustfall Schema Components

    main

    Trustfall datasets are modeled as a graph. A schema defines the shape of data points and their connections. The core components are:

    • Vertices: Typed data items (similar to rows in a SQL table).
    • Properties: Data fields associated with a vertex (similar to columns in a SQL table).
    • Edges: Relationships between different vertex types.
    • Interfaces: Sets of properties that multiple vertex types can implement, allowing for polymorphic queries.
    • Entrypoints: The starting points for queries, defined in a root schema element under the query field. They provide the initial set of vertices to work with.

  4. Define Vertex Types and Properties

    main

    A vertex type is defined using the type keyword. Properties are defined within the type block. Use the ! suffix to denote a required (non-nullable) property.

    Example of a Story vertex with properties:

    """
A story submitted to HackerNews.
"""
type Story {
    """
The story's URL on HackerNews.
"""
    url: String!

    """
The story's title.
"""
    title: String!

    """
The current score of this story submission.
"""
    score: Int!

    """
The URL submitted by this story, if any.
"""
    submittedUrl: String
}

  7. Use parameterized edges

    main

    Parameterized edges accept arguments defined in the schema, which act as predicates.

    Standard Behavior: Without @optional or @recurse, a parameter acts like a @filter directive applied to the edge.

    With @optional: The predicate becomes part of the edge itself. @optional will only consider the edge to exist if it matches the parameter predicate. This differs from a @filter inside an @optional block, where the edge is considered to exist regardless of the filter outcome.

    With @recurse: The predicate is applied to every step of the traversal. If a parameter is provided to a recursive edge, the engine only traverses through edges that satisfy that predicate.

    // Parameterized edge acting as a predicate
{
    Directory {
        out_Directory_ContainsFile(extension: "txt") @optional {
            name @output(out_name: "file_name")
        }
    }
}

  8. Use the `@fold` directive to aggregate edge results into lists

    main

    The @fold directive is applied to edges in a query. It instructs the engine to aggregate the results traversed across that specific edge into a list. This is useful when an edge represents a one-to-many relationship and you want to collect all related values into a single array.

    # in a query, at type `Story`
comment @fold {
    textPlain @output
}

  11. Use the `@transform` directive to count items

    main

    The @transform directive applies a transformation to a value. Currently, the only supported operation is op: "count".

    This directive is typically used in conjunction with @fold and @filter directives on a list of items. When applied to a folded scope, the field defined inside that scope determines which traversed vertices are counted. For example, if you fold over comment objects, the transformation will count the number of comments.

    query {
    Top(max: 20) {
        ... on Story {
            title @output
            comment @fold @transform(op: "count") @output(name: "comment_count") {
                textPlain
            }
        }
    }
}