jsawk

Jsawk is a command-line tool designed for processing JSON data, functioning similarly to awk but specifically for JSON objects. It reads an array of JSON objects from stdin, applies a JavaScript filter to each object, and outputs the resulting array to stdout.

Key capabilities include:

• Filtering and manipulating JSON data using JavaScript snippets.
• Suppressing JSON output to use built-in printing functions for translating JSON into other formats.
• Loading external JavaScript libraries via the command line to extend processing capabilities.
• Integrating into shell pipelines to manipulate data from REST JSON web services.

jsawk is designed to serve the same purpose as awk in a shell environment, but instead of processing lines of text, it processes JavaScript objects and arrays of objects.

In awk, a script runs once for each record (line), and variables like $1, $2 represent fields. In jsawk, the script runs once for each element in the input JSON array (or the object itself if the input is a single object). The current element is accessible via the this object.

Instead of printing modified records to stdout, a jsawk script returns the modified record. If the -n option is not used, the resulting array of returned values is printed as JSON to stdout.

To transform JSON input into modified JSON output, use a script that either modifies the this object in place or returns a new object for each iteration. By default, the modified this object is returned and the final result is printed as a JSON array to stdout.

Examples

Increment an age property:

cat /tmp/t | jsawk 'this.age++'

Flatten an array into a string:

cat /tmp/t | jsawk 'this.sports = this.sports.join(",")'

Extract a single property:

cat /tmp/t | jsawk 'return this.age'

Filter records (JSON Grep): To remove records from the result set, return null.

cat /tmp/t | jsawk 'if (this.age <= 30) return null'

To produce text output instead of JSON, use the -n option to suppress the default JSON result set printing. You can then use out() or err() to print specific values, or rely on the shell to format the output.

Examples

Generate a newline-separated list:

cat /tmp/t \
  | jsawk -a 'return this.join("\n")' 'return this.sports.join("\n")' \
  | sort -u

Check a condition and exit with a status code: Use quit(status) to return a specific exit code to the shell.

jsawk -n 'if (this.age > 50) quit(1)' < /tmp/t || echo "We have people over 50 here"

To install Jsawk, you must first ensure the js interpreter (SpiderMonkey) is installed on your system.

Follow these steps to download and install the jsawk script:

1. Download the script using curl.
2. Make the script executable.
3. Move it to a directory in your PATH (e.g., ~/bin/).

curl -L http://github.com/micha/jsawk/raw/master/jsawk > jsawk
chmod 755 jsawk && mv jsawk ~/bin/

You can specify the path to the Spidermonkey js binary using one of two methods:

1. Use the -j <jsbin> command line option.
2. Set the JS environment variable.

Additionally, jsawk will source the following files at startup if they exist, which can be used to export the JS environment variable:

• _/etc/jsawkrc_
• _~/.jsawkrc_

jsawk allows you to run JavaScript snippets before or after the main processing loop to perform aggregate operations (similar to SQL SUM() or COUNT()).

• Before script (-b <script>): Runs before processing JSON input. The this object is set to the whole JSON array or object. Use this to preprocess the data.
• After script (-a <script>): Runs after the main script has processed all elements. The this object is set to the entire result set. Use this to perform final aggregations or formatting.

Examples

Count elements in the input array:

cat /tmp/t | jsawk -a 'return this.length'

Get a sorted, unique list of all values in a nested array: This example uses an after script to manipulate the result set accumulated during the main loop.

cat /tmp/t \
  | jsawk 'RS=RS.concat(this.sports); return null' -a 'return uniq(RS).sort()'

Jsawk can be used as a filter in a shell pipeline. It takes JSON objects from stdin, executes a JavaScript snippet for each object, and sends the modified objects to stdout.

In the following example, jsawk is used to update a property in a JSON object retrieved from a REST service and then pipe that modified object back to a PUT request.

resty http://example.com:8080/data*.json
GET /people/47 | jsawk 'this.favoriteColor = "blue"' | PUT /people/47