Pathom Documentation

repository·main·Indexed 20 days ago

https://github.com/wilkerlucio/pathom

A Clojure library for writing graph query processing parsers using EQL (EDN Query Language) notation. It features Pathom Connect, a high-level abstraction for building composable graph traversals via resolvers, readers, and an automated indexing system. The library supports mutation joins, asynchronous mutations via async-parser, and integration with Pathom Viz for visual query exploration.

Tokens
38.2K
Snippets
115
Records
150
Agent score
69%

What's inside Pathom

  1. What is Pathom?

    main

    Pathom is a library for building robust parsers to process graph queries for EQL (EDN Query Language) queries. It provides a reader abstraction for easy composition, an entity concept for reusable readers, and a plugin system.

    Key features include:

    • Connect: A high-level abstraction that automatically resolves attribute relationships (e.g., database joins or network requests).
    • Plugin System: Includes built-in plugins for error handling, request caching, and profiling.
    • GraphQL Integration: Ability to use GraphQL endpoints directly from your query system.

    For modern development, it is recommended to use Pathom 3, which is currently in alpha but has a stable API and receives active updates.

  2. What is a Reader in Pathom

    main

    A reader is a function (or a structure acting as a function) that processes a single entry from a query. When a parser encounters a query like [:name :age], it calls a reader for each entry.

    In the case of joins (e.g., [:name :age {:parent [:name :gender]}]), the reader is called once for the join entry itself. The reader is then responsible for checking if there is a child query and performing a recursive call if necessary.

    Readers can take three forms:

    1. Functions: Direct logic to resolve a value.
    2. Maps: Dispatching based on keys.
    3. Vectors: A chain of readers (composition).

    According to Clojure Spec, a reader (::reader) can be a function (::reader-fn), a map of keywords to readers (::reader-map), or a collection of readers (::reader-seq).

  3. What is Pathom and how does it work?

    main

    Pathom is a library designed for building robust parsers to process EDN Query Language (EQL) requests. It is used to fulfill data requests from various sources (databases, micro-services, or GraphQL) on both client and server sides.

    A Pathom parser consists of three main components:

    1. Parser: Receives the full EQL query and iterates over each attribute.
    2. Readers: Process individual entries from the query (properties, joins, or idents). Readers are often organized in a chain; if one reader cannot process a key, it signals the engine to try the next one in the chain.
    3. Resolvers: Used within the Connect feature to traverse a graph of dependencies to resolve data. Each resolver acts as an edge in this graph traversal.

    When running EQL mutations, the mutate function is called, which is typically handled via Connect mutations.

  4. What is Pathom Connect?

    main

    Pathom Connect is a high-level abstraction layer designed for building query processing code. It automates the low-level details of parsing and path resolution, allowing developers to focus on their data model.

    Connect works by generating an index of your graph's features, which enables several automated capabilities:

    • Indexed data traversal: Automatically finds and optimizes paths to fulfill data requirements.
    • Query Auto-completion: Provides auto-complete for graph queries in data exploration tools (such as Pathom Viz).
    • Edge Generation: Automatically generates graph edges based on connection information found in the index.
    • Attribute Resolution: Automatically resolves multiple ways to reach a specific attribute via known reachable edges and transitive relations.
  5. Augment resolvers and mutations with `::pc/transform`

    main

    The ::pc/transform key allows you to wrap a resolver or mutation function with generic operations. This is useful for cross-cutting concerns like database transactions or batching.

    A transform function receives the full resolver/mutation map and must return the modified (or original) map. It can modify any part of the entry, including the ::pc/resolve or ::pc/mutate functions.

    ;; Example: A transform that wraps a mutation in a database transaction
    (defn transform-db-tx [{::pc/keys [mutate] :as mutation}]
      (assoc mutation
        ::pc/mutate
        (fn [env params]
          (db/run-transaction! env #(mutate env params)))));
    
    ;; Applying the transform to a mutation
    (pc/defmutation create-user [env user]
      {::pc/sym       'myapp.user/create
       ::pc/params    [:user/id :user/name]
       ::pc/transform transform-db-tx}
      (db.user/create! env user))
  6. How Connect resolvers and graph edges work together

    main

    Pathom uses a Connect model to process graph queries. The core workflow involves defining resolvers that transform simple inputs (like an entity ID) into a set of outputs.

    Key concepts:

    • Resolvers: Functions that define how to satisfy specific outputs given a set of inputs.
    • Edges: A resolver implies an
    (pc/defresolver person-resolver [env {:keys [person/id] :as params}]
      {::pc/input  #{:person/id}
       ::pc/output [:person/name {:person/address [:address/id]}]}
      {:person/name "Tom" :person/address {:address/id 1}})
  7. Perform joins to traverse graph edges

    main

    A join is used to walk a graph edge to a new entity or set of entities. This is a recursive step where the parser is run on a subquery while replacing the current entity.

    Core Primitive: In its simplest form, a join replaces the current entity in the environment with the new target entity and runs the subquery:

    (defn join [entity {:keys [parser query] :as env}]
      (parser (assoc env ::p/entity entity) query))

    Advanced Usage:

    • p/join: Used to "invent" relations. You can define a computed attribute that, when queried, calls p/join to transition the context to a new entity.
    • * (Wildcard): In a query, * returns all "known" attributes of the current contextual entity.
    • p/join-seq: Similar to p/join, but used for sequences of entities (often used in union queries).
    ; Example: Inventing a join relation
    (def computed
      {:character/voice
       (fn [env]
         (let [{:character/keys [name]} (p/entity env)
               voice (get char-name->voice name)]
           (p/join voice env)))})
    
    (def parser
      (p/parser {::p/plugins [(p/env-plugin {::p/reader [p/map-reader computed]})]}))
    
    (parser {::p/entity rick} 
            '[:character/name
              {:character/voice [:actor/name]}
              {:character/family [* :character/voice]}])
  8. How request caching works

    main

    Request caching in Pathom uses an atom that is initialized at the start of every query. This atom lives within the environment (env) and allows different parts of the query execution to share cached results.

    When p/cached is called, Pathom checks the provided atom for the existence of the specified key. If found, it returns the value; otherwise, it executes the provided function and stores the result in the atom. This is particularly useful for avoiding redundant I/O or heavy computations when multiple branches of a query tree require the same data.

  9. How async parsing works in Pathom

    main

    The async parser allows readers to return core.async channels instead of raw values, enabling parsers to perform asynchronous operations like network requests.

    Key characteristics:

    • Semantic Seriality: Even though it handles async operations, the parser is semantically a serial parser. It preserves the same flow characteristics and resolution order as the regular parser.
    • Compatibility: Core plugins (error handling, profiling, etc.) work normally with the async parser.
    • Performance Note: While the parallel parser is also async, it is generally avoided for most users due to high overhead.

    To create an async parser, use the p/async-parser function.

    ;; Use p/async-parser to define an async parser
    (p/async-parser ...)
  10. Use global resolvers (resolvers without input)

    main

    A resolver with an empty ::pc/input set is a global resolver. It can be invoked at any point in the query graph, not just at the root. This allows you to 'inject' data into any level of the result tree.

    (pc/defresolver latest-product-resolver [_ _]
      {::pc/input {}
       ::pc/output [:product/title]}
      {:product/title "Acoustic Guitar"})
    
    ;; Can be requested anywhere in the graph
    [{::latest-product [:product/title]}]
  11. How the index-oir (Output -> Input -> Resolver) works

    main

    The index-oir is the core index used by the Connect reader to look up attributes. It maps an output attribute to the resolvers that can provide it, keyed by the required input.

    Traversal Logic:

    1. The reader looks up the requested attribute in the index-oir.
    2. It retrieves the set of potential resolvers and their required input sets.
    3. It attempts to match the current entity's attribute keys against these input sets.
    4. If a match is found (e.g., the current context has :id and the resolver requires #{:id}), the resolver is called.
    5. Connect uses <<atom-entities,atom entities>> to merge the resolver's return value back into the context, making the new data available for subsequent attribute lookups.
  12. Compose Readers using Vectors (Vector Dispatcher)

    main

    You can define a chain of readers by using a vector. This is the primary mechanism for reader composition, allowing you to combine modular readers (e.g., from different database modules or libraries) into a single parser.

    How the chain works:

    1. Pathom starts at the first reader in the vector.
    2. If a reader returns a value, the chain stops and that value is used.
    3. If a reader returns ::com.wsscode.pathom.core/continue (::p/continue), Pathom calls the next reader in the vector.
    4. If all readers in the chain return ::p/continue, the final result is ::com.wsscode.pathom.core/not-found (::p/not-found).
    (ns pathom-docs.reader-vector-dispatch
      (:require [com.wsscode.pathom.core :as p]))
    
    ; a map dispatcher for the :name key
    (def name-reader
      {:name   (fn [_] "Saul")})
    
    ; a map dispatcher for the :family key
    (def family-reader
      {:family (fn [_] "Goodman")})
    
    (def composed-reader
      [name-reader
       family-reader])
    
    (def parser (p/parser {::p/plugins [(p/env-plugin {::p/reader composed-reader})]}))
    
    (parser {} [:name :family :other])
    ; => {:name "Saul", :family "Goodman", :other :com.wsscode.pathom.core/not-found}