fx JSON Viewer and Processor

repository·master·Indexed 12 days ago

https://github.com/antonmedv/fx

A command-line tool for processing and viewing JSON, YAML, and TOML data using JavaScript functions. It features an interactive terminal viewer with keyboard navigation, a pipeline pattern for data transformation, and support for custom functions via .fxrc.js. Key features include --slurp for array processing, --raw for non-JSON data, and built-in utilities like uniq, sort, and groupBy.

Tokens
9.1K
Snippets
30
Records
47
Agent score
98%

What's inside fx

  1. How fx processes data using JavaScript functions

    master

    fx treats command-line arguments as JavaScript functions. It implements a pipeline pattern where the input data is passed to the first function, and the result of that function is passed as the input to the next function in the chain.

    Accessing Input Data

    • Arrow Functions: Use an explicit argument like x => x.field.
    • Implicit Input (.): Start an expression with a . to access properties of the input data without writing x => x.
    • this keyword: Use this to refer to the current input data.

    Examples

    Pipeline with arrow functions:

    echo '{"name": "world"}' | fx 'x => x.name' 'x => `Hello, ${x}!`'

    Pipeline with implicit dot and this:

    echo '{"name": "world"}' | fx '.name' '`Hello, ${this}!`'

    Using standard JS functions:

    echo '{"name": "world"}' | fx 'Object.keys'
    echo '{"name": "world"}' | fx 'x => x.name' 'x => `Hello, ${x}!`'
  2. Use syntactic sugar for map and flatMap

    master

    fx provides shorthand syntax to make common functional operations more concise.

    Map Shorthand

    You can use map() without the leading dot or the x => x arrow function syntax.

    Standard: this.map(x => x.commit.message) Sugar: map(.commit.message)

    curl https://api.github.com/repos/antonmedv/fx/commits | fx 'map(.commit.message)'

    FlatMap Shorthand

    For deep traversal or flattening, use the [] syntax.

    Standard: .issues.flatMap(x => x.labels.flatMap(x => x)) Sugar: .issues[].labels[]

    curl https://fx.wtf/example.json | fx '.issues[].labels[]'
    curl https://api.github.com/repos/antonmedv/fx/commits | fx 'map(.commit.message)'
  3. Edit files in-place with the save function

    master

    You can modify the input file directly by using the special save function at the end of your pipeline.

    Example:

    fx file.json 'x.name = x.name.toUpperCase(), x' 'save'

    This will overwrite file.json with the transformed data.

    fx file.json 'x.name = x.name.toUpperCase(), x' 'save'
  4. Configure custom functions via .fxrc.js

    master

    You can extend fx by defining custom functions in a .fxrc.js file. fx looks for this file in:

    1. The current working directory
    2. The user's home directory
    3. The XDG config directory

    Defining functions

    Add your JavaScript functions to the file. To make them available globally in fx, use var instead of let or const.

    Example .fxrc.js content:

    function addOne(x) {
      return x + 1
    }

    Usage in CLI:

    echo '1' | fx addOne
    function addOne(x) {
      return x + 1
    }
    
    echo '1' | fx addOne
  5. Process JSON streams and arrays with --slurp

    master

    By default, fx processes a stream of JSON objects by applying the arguments to each object individually.

    To treat a stream of JSON objects as a single array (slurping them into one collection), use the --slurp or -s flag.

    Individual object processing:

    echo '{"name": "hello"}\n{"name": "world"}' | fx '.name'

    Slurped array processing:

    echo '{"name": "hello"}\n{"name": "world"}' | fx --slurp '.map(x => x.name)' '.join(", ")'
    echo '{"name": "hello"}\n{"name": "world"}' | fx --slurp '.map(x => x.name)' '.join(", ")'
  6. Process non-JSON data with --raw

    master

    To process non-JSON data (like plain text or file lists), use the --raw or -r flag. When using --raw, the input is treated as strings.

    Basic raw processing:

    ls | fx -r '[this, this.includes(".md")]'

    Combining --raw and --slurp: Use -rs to get a single array of strings from a stream of raw lines.

    ls | fx -rs '.filter(x => x.includes(".md"))'

    Using the skip symbol: fx provides a special skip symbol to prevent printing a specific result in the pipeline.

    ls | fx -r '.includes(".md") ? this : skip'
    ls | fx -rs '.filter(x => x.includes(".md"))'
  7. Install fx

    master

    You can install the fx CLI globally via npm, or run it without installation using npx or deno.

    Global installation via npm:

    npm i -g fx

    Using npx:

    cat file.json | npx fx .field

    Using deno:

    cat file.json | deno run -A npm:fx .field
    npm i -g fx
  8. How line navigation works via findNode

    master

    The findNode function implements the logic for locating a specific *Node based on a line number. It traverses the tree structure by prioritizing different node relationships:

    1. Boundary Checks:
      • If the requested line is greater than or equal to m.totalLines, it returns the bottom-most node (m.top.Bottom()).
      • If the requested line is 1 or less, it returns the top node (m.top).
    2. Traversal Logic: The function iterates through the nodes using the following priority for the next node:
      • If node.ChunkEnd is present, it moves to node.ChunkEnd.Next.
      • Otherwise, if node.Collapsed is present, it moves to the node.Collapsed child.
      • Otherwise, it moves to node.Next.
    3. Termination: The loop continues until a node with a matching LineNumber is found or the end of the structure (nil) is reached.
    // Internal traversal pattern used by findNode
    for {
    	if node.ChunkEnd != nil {
    		node = node.ChunkEnd.Next
    	} else if node.Collapsed != nil {
    		node = node.Collapsed
    	} else {
    		node = node.Next
    	}
    
    if node == nil {
    		return nil
    }
    
    if node.LineNumber == line {
    		return node
    }
    }
  9. Search logic and regex behavior in preview

    master

    The preview search uses regular expressions to find matches within the content.

    • Regex Support: The search pattern is treated as a regular expression.
    • Case Sensitivity: The search implementation uses regexCase(pattern) to determine if case-insensitive mode should be applied. If case-insensitive, the (?i) flag is prepended to the regex.
    • Visual Line Calculation: The search engine calculates 'visual line numbers' rather than raw line numbers. This ensures that when you jump to a match, the preview scrolls to the correct position even if the text has been wrapped due to terminal width constraints.
    • Highlighting: Matches are highlighted using a reverseStyle (via lipgloss.NewStyle().Reverse(true)).
  10. Manage YAML anchors and aliases

    master

    YAML anchors (&name) and aliases (*name) allow for data referencing within a document.

    When converting a JavaScript object to a YAML document via toJS(), you can use the onAnchor callback to define how duplicate objects should be handled. If aliasDuplicateObjects is enabled in the context, the library will automatically create anchors for repeated objects and use aliases to reference them in the resulting YAML.

    To resolve an alias back to its anchored node, use Alias.resolve(doc).

  11. How the Composer class works

    master

    The Composer class is the central engine for transforming YAML tokens into a structured document tree. It manages the lifecycle of parsing, including handling directives, preludes (comments/directives before the document), and error/warning collection.

    Key Responsibilities:

    • Directive Management: Uses a Directives instance to track YAML directives (e.g., %YAML 1.2).
    • Error and Warning Collection: Instead of throwing immediately, it collects YAMLParseError and YAMLWarning objects, which can be retrieved via streamInfo() or attached to the resulting Document.
    • Document Composition: The compose generator yields Document objects as they are completed from the token stream.
    • Decoration: The decorate method handles the placement of prelude comments (comments found before the document starts) onto the resulting Document or its first node.

    Usage Pattern:

    Developers typically instantiate a Composer with options and then iterate over the compose generator to process a stream of tokens.

  12. Handling Binary Data in YAML

    master

    The binary tag (tag:yaml.org,2002:binary) allows for the inclusion of binary data.

    Requirements:

    • Reading: Requires either a Buffer (Node.js) or atob (Browser) to decode base64 strings.
    • Writing: Requires either Buffer (Node.js) or btoa (Browser) to encode data to base64.

    When writing binary data, the library can format it as a BLOCK_LITERAL or a quoted string, respecting the lineWidth configuration to wrap lines appropriately.