DOCSEARCHbyAlgolia
Submit Docs

relic

repository·master·Indexed 19 days ago

https://github.com/wotbrew/relic

A Clojure(Script) in-memory database implementing a functional relational programming model. It features a vector-based DSL for SQL-style queries, incremental materialized views via a data flow graph, relational constraints using :check, and reactive change tracking with rel/watch and rel/track-transact. It supports advanced aggregations (:agg), sorted indexing (:btree), and integration with standard Clojure functions.

Tokens
19.4K
Snippets
83
Records
100
Agent score
64%

What's inside relic

  1. Core features of relic

    master

    Relic provides several key capabilities for in-memory relational data management:

    • Indexed SQL-style queries: Perform complex queries using a vector-based DSL.
    • Clojure Integration: Use standard Clojure functions within your queries.
    • Incremental Materialized Views: Automatically maintain query results as data changes.
    • Constraints: Define relational constraints to make invalid states illegal.
    • Reactive Change Tracking: Efficiently integrate with UI frameworks (like React) by binding components to materialized queries.

  2. Materialize indexes for query optimization

    master
    Relic allows you to materialize indexes to enable query optimizations or to provide direct index access for high-performance tasks. While Relic automatically creates indexes for operations like :join, :left-join, and :fk, you can manually define specific index types for specialized needs.

  3. How sub queries work under the hood

    master
    Sub queries in relic are not executed as independent queries for every expression evaluation. Instead, they are converted into implicit join dependencies. This allows them to utilize indexes in the same way standard joins do, ensuring performance efficiency.

  4. How queries are structured in relic

    master

    In relic, a query is represented as a vector of operations. Each operation is itself a vector in the form [operator & args].

    Data flows top-to-bottom through the query. Unlike many query optimizers that reorder operations, relic queries compose by adding operations to the vector, meaning the order you define them in is the order they are executed. This allows you to use standard Clojure functions like conj to add operations or split-at to manipulate the query structure.

    ;; A query is a vector of operations
[[:from :table]
 [:where [= :col 1]]
 [:select [:col]]]

  5. Security considerations for untrusted input in relic

    master

    Because relic supports Clojure functions in queries (e.g., [sh/sh "rm" "-rf" "/"]), untrusted user input can lead to Remote Code Execution (RCE) or data exfiltration if not handled carefully.

    Injection Vectors

    • Prefix Position: Relic restricts what can be placed in the prefix position of an expression to mitigate injection. Allowed types include function objects (not symbols), whitelisted sentinel values, safe keywords like :_, and column keywords.
    • Data Exfiltration: While column keywords do not provide RCE, they can be used to exfiltrate row data. For example, an expression like [:extend [:my-col untrusted]] where untrusted is user-controlled could be used to access unauthorized data.
    • Denial of Service (DoS): Allowing users to create arbitrary keywords can pollute the keyword cache, increasing memory pressure and collection time, potentially leading to a DoS.

  6. How constraints work in relic

    master

    Constraints are queries that end with a specific constraint statement. When a constraint is violated, relic throws an exception. They are used to ensure databases remain in valid states by preventing invalid data from being committed or processed.

    To enforce constraints on a database, you must materialize the constraint queries. You can remove them using demat.

    [[:from Order] 
 [:join Customer {:customer-id :customer-id}]
 [:where [= "bob" :firstname] [tuesday? [rel/env :now]]]
 [:check {:pred [<= [count :items] 10],
          :error [str "order can have at most 10 items if its associated customer is called bob and its tuesday, found: " [count :items]]}]]

  7. Manage global parameters using the environment

    master

    In relic, global parameters that affect query results but are not part of the query definition itself (such as the current time or environment variables) should be stored in a special table called the environment. This prevents the need to pass these parameters through every query and provides a centralized way to manage state that is external to your relational data.

    Key operations for managing the environment include:

    • Replacing the entire environment map.
    • Retrieving the current environment map.
    • Updating specific keys within the environment map.
    • Referencing environment values within queries using the [rel/env :key] expression form.
    ;; Replace the environment with a new map
(rel/with-env db {:now (System/currentTimeMillis)})

;; Reference the environment in a query
[[:select [:seconds [/ [rel/env :now] 1000]]]]

;; Get the current environment map
(rel/get-env db)

;; Update the environment using a function (e.g., assoc)
(rel/update-env db assoc :now (System/currentTimeMillis))

  8. What are incremental materialized views in relic?

    master

    Relic is powered by a data flow graph. Instead of just evaluating a query once, you can use rel/mat to create an incremental materialized view.

    When you call rel/mat on a query, it returns a new database instance that contains the necessary internal machinery to maintain that query's result automatically. As you modify the underlying tables via transact, the changes flow through the graph and update the materialized view instantly. This allows for highly efficient, reactive data processing.

    ;; Materializing a query
(rel/mat db [[:from :Customer] [:where [= :name "bob"]]])
;; Returns a new database where this specific query is maintained incrementally.

  9. Use the :agg operator for aggregate functions

    master

    The :agg operator allows you to perform aggregate operations on data sets, such as calculating sums, counts, or finding maximum/minimum values. These operations are typically applied over a collection of rows based on specific expressions or predicates.

    ;; Example usage pattern (conceptual)
{:agg [:count :predicate]}

  11. Choose between Clojure functions and relic expressions

    master

    relic supports two ways to perform computation and define conditions within queries. You can use standard Clojure functions for arbitrary, complex logic, or use the relic expression DSL for simple, inline computations.

    When to use Clojure functions

    Use Clojure functions when you need complex logic that is difficult to express in the DSL. Critical Requirement: Functions used as expressions must be referentially transparent (pure) to avoid glitches in the reactive system.

    When to use relic expressions

    Use the relic expression DSL for:

    • Simple inline computations.
    • Better ergonomics when working with rows (keywords are automatically substituted for lookups).
    • Avoiding Clojure's quote/unquote template syntax.
    • Utilizing built-in features like sub-queries, nil-safe function application, and special conditional forms.

    Where expressions can be used

    Expressions can be applied in the following query clauses:

    • [:where ...] conditions
    • [:extend ...] extensions
    • [:select ...] projections
    • [:expand ...] expansions
    • [:join ...] and [:left-join ...] clauses
    • [:agg ...] (certain aggregates like sum accept expressions as arguments)
    • [:hash ...] and [:btree ...] indexed expressions
    ;; Example of using a Clojure function
(defn my-pred? [{:keys [foo]}] (= foo 42))
[[:from :A] [:where my-pred?]]

;; Example of using a relic expression
[[:from :A] [= :foo 42]]

  12. Behavior of :agg with empty relations

    master
    When using :agg to group over all rows (using an empty vector [] for cols), the operator will always return a row even if the input relation is empty. It will not return nil. For example, if you request a count of all rows on an empty relation, you will receive {:count 0}.