DatalogUI Datalog

repository·master·Indexed 41 days ago

https://github.com/datalogui/datalog

A Differential Datalog implementation in JavaScript optimized for managing UI state and reactive data updates. It features a typed schema, an expressive querying syntax that supports differential updates, and the ability to run queries on the results of previous queries. It includes core primitives for defining tables, views, and relations, and integrates with @datalogui/react.

Tokens
4.9K
Snippets
16
Records
21
Agent score
49%

What's inside DatalogUI

  1. What is @datalogui/datalog?

    master
    Datalog is a Differential Datalog implementation in JavaScript designed with a focus on managing UIs and UI state. It provides an expressive querying syntax that supports differential updates—meaning queries only run on data differences rather than the entire dataset every time. It also supports 'Querying your Queries,' allowing you to run new queries on the results of previous ones. It features a typed schema and integrates with @datalogui/react.
  2. Create tables and run queries in Datalog

    master

    To use Datalog, you first define tables using newTable<T>() with a schema defining the types of your fields. You can then populate these tables using .assert(). To retrieve data, use query<T>() which takes a function defining the relational logic. The query returns a view that can be read using .view().readAllData().

    import * as datalog from '@datalogui/datalog'
    
    // 1. Define a table with a schema
    const People = datalog.newTable<{ id: number, name: string }>({
        id: datalog.NumberType,
        name: datalog.StringType,
    })
    
    // 2. Add data using .assert()
    People.assert({id: 0, name: "Alice"})
    People.assert({id: 1, name: "Charles"})
    
    // 3. Define a relation table
    const ParentOf = datalog.newTable<{ parentID: number, childID: number }>({
        parentID: datalog.NumberType,
        childID: datalog.NumberType,
    })
    ParentOf.assert({parentID: 1, childID: 0})
    
    // 4. Define a query
    const Query = datalog.query<{parentName: string, parentID: number, childID: number}>(({parentName, parentID, childID}) => {
      People({name: "Alice", id: childID})
      ParentOf({childID, parentID})
      People({id: parentID, name: parentName})
    })
    
    // 5. Read the results
    const results = Query.view().readAllData()
    // => [{childID: 0, parentID: 1, parentName: "Charles"}, ...]
  3. Use `MaterializedTable` for reactive queries

    master

    A MaterializedTable is the result of a query. It provides several methods for interacting with the derived data:

    • runQuery(): Manually triggers a re-run of the query logic.
    • onDependencyChange(subscriber): Returns an unsubscribe function; calls the subscriber whenever any input table changes.
    • view(): Returns a View<T> for reading the current state.
    • implies(f): Adds recursive implications. The function f is called whenever the materialized table's data changes (either added or removed), allowing the results of one query to feed back into other tables.
    const results = query((f) => {
      users(f);
    });
    
    // React to changes in the query results
    const unsubscribe = results.onDependencyChange(() => {
      console.log('Query dependencies changed, new data:', results.readAllData());
    });
    
    // Add recursive logic
    results.implies((datum, kind) => {
      if (kind === Added) {
        console.log('New result found:', datum);
      }
    });
  4. Query the results of another query

    master

    Datalog allows you to treat the result of a query as if it were a table. You can pass a previously defined query object into a new query<T>() block to filter or transform its results.

    // Assuming 'Query' was defined previously
    
    // Give me the ID of anyone named "Helen" from the results of 'Query'
    const QueryQuery = datalog.query<{parentID: number}>(({parentID}) => {
      Query({parentID, parentName: "Helen"})
    })
    
    const results = QueryQuery.view().readAllData()
  5. Perform queries with `query`

    master

    The query function allows you to define a Datalog query using a callback function. Inside the callback, you use a FreeVarGenerator to map table fields to query variables. The result is a MaterializedTable that automatically updates when its dependencies change.

    import { query, intoTable, FreeVarGenerator } from '@datalogui/datalog';
    
    const users = intoTable([{ name: 'Alice', age: 30 }, { name: 'Bob', age: 25 }]);
    
    const adults = query((f) => {
      // f.name and f.age are free variables mapped to the users table
      users(f);
      // You can add logic here to filter or join
    });
    
    // adults is a MaterializedTable
    console.log(adults.readAllData());
  6. Perform joins using variableJoinHelper

    master

    The variableJoinHelper function is used to perform joins between multiple Variable instances. It allows you to define logic that executes whenever the joined result changes.

    It supports joining 1, 2, or 3 variables. You can provide remapKeys to transform the output shape and constants to filter the join results.

    Signature (for 2 variables): variableJoinHelper(logicFn, variables, remapKeys, constants)

    • logicFn: A function called with the joined object and an isRetraction boolean.
    • variables: An array containing the Variable instances to join.
    • remapKeys: An array of mapping objects to reshape the output.
    • constants: An array of partial objects used to filter the join.
    // Example conceptual usage for joining two variables
    // variableJoinHelper(
    //   (joined, isRetraction) => { ... }, 
    //   [varA, varB], 
    //   [remapA, remapB], 
    //   [constA, constB]
    // );
  7. Manage data with the Relation class

    master

    The Relation<T> class is a multi-indexed collection of data of type T. It supports assertions (adding data) and retractions (removing data). It maintains multiple indices to allow efficient lookups by different key orderings.

    Key operations:

    • assert(v: T): Adds a datum to the relation.
    • retract(v: T): Removes a datum from the relation.
    • indexBy(ks: [keyof T, ...Array<keyof T>]): Creates and returns a RelationIndex optimized for the specified key order.
    • indexByK(k: keyof T): Creates and returns a RelationIndex optimized for a single key k as the primary index.
    • merge(otherRelation: Relation<T>): Merges another relation into this one, handling both assertions and retractions.
    import { Relation } from './datalog';
    
    interface User {
      id: number;
      name: string;
      email: string;
    }
    
    const users = new Relation<User>();
    
    // Asserting data
    users.assert({ id: 1, name: 'Alice', email: 'alice@example.com' });
    
    // Creating an index by 'id'
    const idIndex = users.indexByK('id');
    
    // Retracting data
    users.retract({ id: 1, name: 'Alice', email: 'alice@example.com' });
  8. Understand RelationIndex and key ordering

    master

    A RelationIndex<KName, K, Val> is a sorted, indexed representation of a relation's elements. It is the underlying structure used for efficient searching and joining.

    • elements: An array of tuples representing the rows. Rows can be marked as retractions using a internal MetaSymbol.
    • keyOrdering: An array defining the sequence of keys used for sorting and indexing.
    • dedup(): Removes duplicate elements from the index.
    • indexBy(...): Creates a new RelationIndex with a different key ordering, effectively re-indexing the data.
    • filterElements(constants, isAnti): Returns a new RelationIndex containing only rows that match the provided constants. If isAnti is true, it returns rows that do not match.
  9. Transform data with ViewExt methods

    master

    The ViewExt<T> interface extends the base View<T> to provide functional transformations on data streams. You can use these methods to derive new views from existing ones:

    • map<O>(f: (t: T) => O): Creates a new view where each element is transformed by function f.
    • mapEffect<F>(f: F): Executes a side effect for every change in the view. This returns a SingleItemView<{}> used primarily to signal when effects have occurred.
    • reduce<Acc>(reducer, initialVal): Accumulates values over time using a reducer function, returning a ViewExt<Acc> containing the current state of the accumulator.
    • sortBy(sortFn): Returns an IndexedViewExt<T> where elements are ordered by sortFn and assigned an index.
    • orderBy(key, ascending?): A convenience method for sortBy that orders elements by a specific object key.
    // Example of mapping and reducing a view
    const mappedView = view.map(item => item.value);
    const sumView = view.reduce((acc, recent) => {
      if (recent.kind === datalog.Added) return acc + recent.datum;
      return acc;
    }, 0);
    
    // Example of sorting
    const sortedView = view.sortBy((a, b) => a.name.localeCompare(b.name));
  10. Manage indexed data with IndexedViewExt

    master

    When working with ordered data, IndexedViewExt<T> provides methods to slice and transform views that contain Indexed<T> objects (objects with an index and the original datum).

    • take(n): Returns a new view containing only the first n elements.
    • drop(n): Returns a new view that skips the first n elements.
    • mapIndexed<O>(f: (t: T) => O): Transforms the underlying data while preserving the indexing structure, returning an IndexedViewExt<O>.
    • sortBy(sortFn) and orderBy(key, ascending?): Re-sorts the indexed view.
    // Slicing an indexed view
    const topFive = indexedView.take(5);
    const skippedView = indexedView.drop(10);
    
    // Mapping data within an indexed view
    const nameView = indexedView.mapIndexed(item => item.name);
  11. Create a new table with `newTable`

    master

    Use newTable to create a base table with a defined schema. The schema uses TypeValidator objects to ensure data integrity. Supported validators include StringType, NumberType, BoolType, ObjectType, and ArrayType.

    import { newTable, StringType, NumberType } from '@datalogui/datalog';
    
    const schema = {
      name: StringType,
      age: NumberType
    };
    
    const users = newTable(schema);
    
    users.assert({ name: 'Alice', age: 30 });
  12. Track changes with the Variable class

    master

    The Variable<T> class represents a dynamic data source that tracks changes over time. It is designed for differential computation, maintaining a stable state (the current ground truth) and a recent state (the changes since the last check).

    Key features:

    • assert(v: T) / retract(v: T): Updates the variable. It tracks counts to determine if a datum is truly new or being removed.
    • onChange(f: () => void): Subscribes to any change in the variable. Returns a cleanup function.
    • onNewDatum(f: (d: RecentDatum<T>) => void): Subscribes to specific additions or removals. Returns a cleanup function.
    • changed(): boolean: Merges recent changes into stable and returns true if there were pending changes.
    • recentData(): Returns an array of RecentDatum<T> objects containing the kind (Added or Removed) and the datum itself.
    import { Variable, Added, Removed } from './datalog';
    
    interface Point {
      x: number;
      y: number;
    }
    
    const points = new Variable<Point>();
    
    // Subscribe to new data
    const unsubscribe = points.onNewDatum((change) => {
      if (change.kind === Added) {
        console.log('New point:', change.datum);
      } else if (change.kind === Removed) {
        console.log('Point removed:', change.datum);
      }
    });
    
    points.assert({ x: 10, y: 20 });
    
    // Cleanup subscription
    unsubscribe();