liqe

repository·main·Indexed 20 days ago

https://github.com/gajus/liqe

A lightweight and performant Lucene-like parser, serializer, and in-memory search engine for JSON documents using the Liqe Query Language (LQL). It provides functions to filter collections, test objects, and highlight matches, as well as utilities to parse LQL strings into an Abstract Syntax Tree (AST) and serialize ASTs back into query strings.

Tokens
3.3K
Snippets
13
Records
13
Agent score
71%

What's inside liqe

  1. The Liqe AST (Abstract Syntax Tree)

    main

    The Liqe AST is a public API. Developers can implement their own search engines or custom logic by traversing the 11 available token types. If you are building a custom serializer, you must implement all 11 tokens to ensure complete coverage of the LQL syntax.

    import {
      type BooleanOperatorToken,
      type ComparisonOperatorToken,
      type EmptyExpression,
      type FieldToken,
      type ImplicitBooleanOperatorToken,
      type ImplicitFieldToken,
      type LiteralExpressionToken,
      type LogicalExpressionToken,
      type RangeExpressionToken,
      type RegexExpressionToken,
      type TagToken,
      type UnaryOperatorToken,
    } from 'liqe';
  2. Use Liqe to filter collections, test objects, and highlight matches

    main

    Liqe provides high-level functions to perform in-memory searches on JSON documents using the Liqe Query Language (LQL).

    • filter(parsedQuery, collection): Returns a subset of the collection that matches the query.
    • test(parsedQuery, object): Returns true if the specific object matches the query, otherwise false.
    • highlight(parsedQuery, object): Returns an array of match metadata, including the path to the field and a query (RegExp) for the matching substring, useful for UI highlighting.
    import {
      filter,
      highlight,
      parse,
      test,
    } from 'liqe';
    
    const persons = [
      { height: 180, name: 'John Morton' },
      { height: 175, name: 'David Barker' },
      { height: 170, name: 'Thomas Castro' },
    ];
    
    // Filter a collection
    filter(parse('height:>170'), persons);
    
    // Test a single object
    test(parse('name:John'), persons[0]); // true
    
    // Highlight matches
    highlight(parse('name:john'), persons[0]);
    // [
    //   { path: 'name', query: /(John)/ }
    // ]
  3. Handle Liqe syntax errors

    main

    When an invalid query is provided to parse(), Liqe throws a SyntaxError. This error object contains metadata about where the error occurred, including message, offset, line, and column.

    import {
      parse,
      SyntaxError,
    } from 'liqe';
    
    try {
      parse('foo bar');
    } catch (error) {
      if (error instanceof SyntaxError) {
        console.error({
          message: error.message,
          offset: error.offset,
          line: error.line,
          column: error.column,
        });
      } else {
        throw error;
      }
    }
  4. Serialize parsed tokens back to LQL string

    main

    The serialize function converts a parsed AST (tokens) back into its original Liqe Query Language (LQL) string representation. This is useful when you need to programmatically manipulate the query structure before converting it back to text.

    import {
      parse,
      serialize,
    } from 'liqe';
    
    const tokens = parse('foo:bar');
    serialize(tokens); // 'foo:bar'
  5. Determine if an expression requires quotes

    main

    The isSafeUnquotedExpression utility helps determine if a string expression can be safely written without quotes. This is useful when programmatically manipulating the AST before using a serializer to convert the query back to text.

    import {
      isSafeUnquotedExpression,
    } from 'liqe';
    
    isSafeUnquotedExpression(expression: string): boolean;
  6. Liqe Query Language (LQL) Syntax Cheat Sheet

    main

    LQL is a Lucene-inspired language for searching JSON documents.

    Keyword & Phrase Matching

    • foo: Case-insensitive search for "foo" anywhere in the document.
    • 'foo' or "foo": Case-sensitive search for "foo".
    • name:foo: Search for "foo" in the name field.
    • 'full name':foo: Search for "foo" in a field with spaces.
    • name.first:foo: Search for a nested field (e.g., {name: {first: 'foo'}}).

    Regex & Wildcards

    • name:/foo/: Regex search.
    • name:/foo/o: Regex search with specific flags.
    • name:foo*bar: Wildcard search (* for multiple characters).
    • name:foo?bar: Wildcard search (? for a single character).

    Numbers & Ranges

    • height:=100: Exact match.
    • height:>100, height:>=100, height:<100, height:<=100: Comparison operators.
    • height:[100 TO 200]: Inclusive range.
    • height:{100 TO 200}: Exclusive range.

    Boolean Operators

    • AND, OR, NOT, -: Standard boolean logic.
    • name:foo height:=100: Implicit AND operator.
    • (expression): Grouping expressions.

    Special Values

    • member:true / member:false: Boolean search.
    • member:null: Null search.
    # search for "foo" term anywhere in the document (case insensitive)
    foo
    
    # search for "foo" term anywhere in the document (case sensitive)
    'foo'
    "foo"
    
    # search for "foo" term in `name` field
    name:foo
    
    # search for "foo" term in `full name` field
    'full name':foo
    "full name":foo
    
    # search for "foo" term in `first` field, member of `name`, i.e.
    # matches {name: {first: 'foo'}}
    name.first:foo
    
    # search using regex
    name:/foo/
    name:/foo/o
    
    # search using wildcard
    name:foo*bar
    name:foo?bar
    
    # boolean search
    member:true
    member:false
    
    # null search
    member:null
    
    # search for age =, >, >=, <, <=
    height:=100
    height:>100
    height:>=100
    height:<100
    height:<=100
    
    # search for height in range (inclusive, exclusive)
    height:[100 TO 200]
    height:{100 TO 200}
    
    # boolean operators
    name:foo AND height:=100
    name:foo OR name:bar
    
    # unary operators
    NOT foo
    -foo
    NOT foo:bar
    -foo:bar
    name:foo AND NOT (bio:bar OR bio:baz)
    
    # implicit AND boolean operator
    name:foo height:=100
    
    # grouping
    name:foo AND (bio:bar OR bio:baz)
  7. Filter an array of objects using a Liqe query

    main

    Use the filter function to select elements from a readonly array of objects that match a given LiqeQuery AST. This function takes a parsed Liqe query (AST) and a data array, returning a new readonly array containing only the elements that satisfy the query criteria.

    import { filter } from './filter';
    
    const data = [
      { name: 'Alice', age: 30 },
      { name: 'Bob', age: 25 },
      { name: 'Charlie', age: 35 }
    ];
    
    // Assuming 'ast' is a valid LiqeQuery AST
    const results = filter(ast, data);
  8. Highlight matching parts of data using highlight()

    main

    The highlight function identifies matching parts of a data object based on a provided LiqeQuery AST. It returns an array of Highlight objects, where each object specifies a path (the location in the data object) and a query (a regular expression used to match the keywords found at that path). This is useful for UI components that need to visually emphasize search terms within search results.

    Parameters:

    • ast: A LiqeQuery representing the search query structure.
    • data: The data object T to be scanned for matches.

    Returns: An array of Highlight objects. Each object contains:

    • path: A string representing the path to the matched data.
    • query (optional): A RegExp constructed from the keywords found at that path, used to identify the exact substrings to highlight.
    import { highlight } from './highlight';
    
    // Example usage (conceptual):
    // const highlights = highlight(queryAst, dataObject);
    // highlights.forEach(({ path, query }) => {
    //   if (query) {
    //     console.log(`At ${path}, match with regex: ${query}`);
    //   }
    // });
  9. Convert a Liqe AST to a query string with serialize()

    main

    The serialize function converts a LiqeQuery Abstract Syntax Tree (AST) back into its original Liqe query string representation. This is useful for debugging, logging, or reconstructing queries after programmatic manipulation of the AST.

    Supported AST types include:

    • Tag: Represents a field-level operation (e.g., field:value).
    • LogicalExpression: Combines expressions using boolean operators (e.g., AND, OR).
    • UnaryOperator: Handles operators like NOT.
    • ParenthesizedExpression: Wraps expressions in parentheses ().
    • EmptyExpression: Returns an empty string.

    Note: For ParenthesizedExpression and LogicalExpression, the serializer relies on location metadata within the AST to preserve the original whitespace and formatting.

    import { serialize } from './serialize';
    
    // Assuming 'ast' is a valid LiqeQuery object
    const queryString = serialize(ast);
    console.log(queryString);
  10. Parse a Liqe query string using parse()

    main

    Use the parse function to convert a raw Liqe query string into a structured LiqeQuery AST (Abstract Syntax Tree).

    If the input string is empty or contains only whitespace, it returns an EmptyExpression type with zero-based start and end locations. If the query is syntactically invalid, it throws a SyntaxError containing the specific line and column where the error occurred. If the parser produces multiple different valid interpretations of the same string, it throws a LiqeError with the message Ambiguous results..

    import { parse } from './parse';
    
    try {
      const ast = parse('name.first: "foo"');
      console.log(ast);
    } catch (error) {
      if (error instanceof SyntaxError) {
        console.error(`Error at line ${error.line}, col ${error.column}: ${error.message}`);
      } else {
        console.error(error);
      }
    }
  11. Reference the Liqe boolean operators

    main

    Logical operations in Liqe queries use the following boolean operators:

    • AND (Explicit)
    • OR (Explicit)
    • AND (Implicit, e.g., a space between terms like foo bar)
    export type BooleanOperatorToken = {
      location: TokenLocation;
      operator: 'AND' | 'OR';
      type: 'BooleanOperator';
    };
    
    export type ImplicitBooleanOperatorToken = {
      operator: 'AND';
      type: 'ImplicitBooleanOperator';
    };
  12. Reference the Liqe query comparison operators

    main

    When constructing or analyzing Liqe queries, the following comparison operators are supported for field matching:

    • : (Equality/Match)
    • :< (Strictly less than)
    • :<= (Less than or equal to)
    • := (Assignment/Equality)
    • :> (Strictly greater than)
    • :>= (Greater than or equal to)
    export type ComparisonOperator = ':' | ':<' | ':<=' | ':=' | ':>' | ':>=';