yq Command-Line Processor

repository·master·Indexed 12 days ago

https://github.com/mikefarah/yq

A lightweight, portable command-line processor for YAML, JSON, INI, and XML that provides a jq-like syntax for querying and manipulating structured data files. It supports in-place updates, format conversion, and merging multiple files using the eval-all (ea) command.

Tokens
56.7K
Snippets
361
Records
453
Agent score
96%

What's inside yq

  1. What is KYaml and when to use it

    master

    KYaml is a restricted subset of YAML that uses flow-style collections. It is designed to render YAML data in a compact, JSON-like form while retaining YAML-specific features such as comments.

    When using KYaml:

    • Strings are always output with double quotes.
    • Anchors and aliases are expanded into their full values (KYaml does not emit anchors or aliases in its output).
  2. What is KYaml and how to use it

    master

    KYaml is a restricted subset of YAML that uses flow-style collections (braces {} and brackets []). It is useful for rendering YAML data in a compact, JSON-like format while still supporting YAML features like comments.

    Key characteristics of KYaml output:

    • Strings are always double-quoted.
    • Anchors and aliases are expanded into concrete values (they are not emitted as anchors/aliases).
    • It uses flow-style collections and explicit commas.
    • It supports arbitrary nesting of lists and objects.
    yq -o=kyaml '.' sample.yml
  3. Pipe the results of an expression into another

    master

    Use the pipe operator (|) to pass the output of one yq expression as the input to the next. This functions similarly to the pipe operator in bash, allowing you to chain multiple transformations together in a single command.

    # Example of chaining operations
    yq '.foo | .bar' file.yaml
  4. Handle illegal variable names when encoding to shell

    master

    When using -o=shell, yq automatically adapts keys that would be illegal in a shell environment:

    • Symbols: Characters like = are replaced with _.
    • Whitespace/Controls: Keys containing tabs or other control characters are dropped.
    • Non-ASCII: Non-ASCII characters are dropped.
    • Accented Characters: Accented Latin letters are normalized via Unicode NFKD (e.g., ñ becomes n).
  5. Traverse YAML aliases and merge anchors

    master

    Aliases (*anchor) and merge anchors (<<: *anchor) can be traversed.

    • To traverse the content of an alias, use the splat operator: .b[].
    • To traverse a specific key within an alias, use explicit paths: .b.c.
    • When using merge anchors, the behavior of key overrides depends on whether --yaml-fix-merge-anchor-to-spec is enabled.
    # Traversing aliases with splat
    yq '.b[]' sample.yml
    
    # Traversing aliases explicitly
    yq '.b.c' sample.yml
  6. Use Encoder and Decoder operators

    master

    Encoder operators transform an object structure into a formatted string. Decoder operators perform the inverse, converting a formatted string back into an object structure. These are useful for processing YAML documents that contain embedded stringified data (like JSON, XML, or CSV).

    FormatDecode (from string)Encode (to string)
    Yamlfrom_yaml / @yamldto_yaml(i) / @yaml
    JSONfrom_json / @jsondto_json(i) / @json
    Propertiesfrom_props / @propsdto_props / @props
    CSVfrom_csv / @csvdto_csv / @csv
    TSVfrom_tsv / @tsvdto_tsv / @tsv
    XMLfrom_xml / @xmldto_xml(i) / @xml
    Base64@base64d@base64
    URI@urid@uri
    Shell@sh
  7. Use boolean operators (and, or, not)

    master

    The and and or operators take two parameters and return a boolean result. The not operator flips a boolean value (true to false, or vice versa).

    Note that in yq, non-boolean values have truthiness:

    • Strings (including empty strings "") are considered true.
    • Numbers (including 0) are considered true.
    • null (~) is considered false.
    # or example
    yq --null-input 'true or false'
    
    # and example
    yq --null-input 'true and false'
    
    # not examples
    yq --null-input 'true | not'
    yq --null-input 'false | not'
    yq --null-input '~ | not'