Jolt Documentation

Jolt is a JSON-to-JSON transformation library written in Java. It uses a JSON document as a "specification" to define how data should be transformed.

Key characteristics:

• Structural Focus: Jolt is designed to transform the structure of JSON data rather than manipulating specific values. The recommended pattern is to use Jolt for structural changes and use custom Java code for value manipulation.
• Chained Transforms: You can chain multiple transforms together to form a complete transformation pipeline.
• Hydrated JSON: Jolt operates on "hydrated" JSON, which is an in-memory tree of Java objects (Maps, Lists, Strings, etc.). You should use a library like Jackson to handle the initial serialization/deserialization of JSON text into these objects.

While Jolt is optimized for developer speed through declarative transforms, users should be aware of the following performance characteristics:

• Memory Usage: Jolt is not stream-based. You must have enough memory to hold the entire JSON document in memory as a hydrated tree of Java objects.
• Garbage Collection: The transformation process creates and discards many objects, which increases the workload for the Java Garbage Collector.
• Thread Safety: Transforms can be initialized once with their spec and then re-used many times in a multi-threaded environment (e.g., within a web service).

Jolt provides several built-in "Stock" transforms, each with its own Domain Specific Language (DSL) to perform specific structural tasks:

• shift: Copies data from the input tree to the output tree.
• default: Applies default values to the tree.
• remove: Removes data from the tree.
• sort: Sorts Map key values alphabetically (useful for debugging).
• cardinality: Fixes the cardinality of input data (e.g., ensuring an element is always a List even if it only contains one item).

For complex data manipulation that these structural transforms cannot handle, you can implement the Transform or ContextualTransform interfaces in Java and insert your custom logic into the transform chain.

The shift transform (often referred to as Shiftr) is typically the most important part of a Jolt specification. It handles the heavy lifting of moving data from one structure to another.

To learn the Shiftr DSL, it is recommended to examine the relationship between input, spec, and expected output in Jolt's unit tests. The pattern used in tests is:

{
    "input": { /* sample input */ },
    "spec": { /* transform spec */ },
    "expected": { /* what the output should look like */ }
}

By comparing the input and expected JSON, you can deduce how the spec facilitates the movement of data.

To use Jolt in your Java project, add the following Maven dependencies to your pom.xml. You should replace ${latest.jolt.version} with the current version found in the project's releases (e.g., 0.0.16).

There are two primary artifacts:

1. jolt-core: A pure Java artifact with minimal dependencies (only apache.commons for StringUtils) to avoid dependency conflicts.
2. json-utils: A Jackson wrapper and testing utility. Use this if you are willing to include Jackson 2 in your project to access convenient utility methods.

<dependency>
    <groupId>com.bazaarvoice.jolt</groupId>
    <artifactId>jolt-core</artifactId>
    <version>${latest.jolt.version}</version>
</dependency>
<dependency>
    <groupId>com.bazaarvoice.jolt</groupId>
    <artifactId>json-utils</artifactId>
    <version>${latest.jolt.version}</version>
</dependency>

To use the Jolt CLI, you must build it from the source and add the binary directory to your system's PATH.

1. Clone the repository.
2. Build the project using Maven: mvn clean package.
3. Add the bin/ directory from the project root to your PATH environment variable.

cd $JOLT_CHECKOUT
git pull
mvn clean package
# Then add $JOLT_CHECKOUT/bin/ to your PATH

The Jolt CLI supports standard input (stdin), allowing you to chain commands or pipe data from web requests directly into Jolt operations.

Example: Transform and then Sort

curl -s "http://api.example.com/data" | jolt transform spec.json | jolt sort

Example: Diff an API response against a local file

curl -s "http://api.example.com/data.json" | jolt diffy local_data.json

curl -s "http://some.json.api.com/data/you/will/take/our/stupid/format/and/like/it" | jolt transform makeSaneSpec.json | jolt sort
curl -s "http://some.host.com/stuff/data.json" | jolt diffy moreData.json

To run a sequence of transforms, you use the Chainr class. You provide it with a JSON specification that defines the list of transforms to be applied.

In Java, you can load the specification using JsonUtils.classpathToList and then call .transform(input) on the Chainr instance.

Chainr chainr = JsonUtils.classpathToList( "/path/to/chainr/spec.json" );

Object input = elasticSearchHit.getSource(); // ElasticSearch already returns hydrated JSON

Object output = chainr.transform( input );

return output;

The diffy subcommand compares two JSON documents to detect differences.

Exit Codes:

• 0: No differences found
• 1: Differences found or an error encountered

Usage: jolt diffy [-h] [-s] [-a] [-i] filePath1 [filePath2]

Arguments:

• filePath1: Path to the first JSON file.
• filePath2: Path to the second JSON file. This is mutually exclusive with the -i flag.

Options:

• -h, --help: Show help message.
• -s: Run silently (suppresses output).
• -a: Ignore array order when detecting differences.

jolt diffy input1.json input2.json

The sort subcommand sorts a JSON document. The sort order is standard alphabetical ascending, except that keys prefixed with ~ are bumped to the top.

Exit Codes:

• 0: Success
• 1: Error encountered

Usage: jolt sort [-h] [-u] [input]

Arguments:

• input: Path to the JSON input file. If omitted, the tool reads from stdin.

Options:

• -h, --help: Show help message.
• -u: Turns off pretty printing. Output will be raw JSON without formatting (default is false).

jolt sort input.json

The transform subcommand executes a Jolt Transform Spec against a JSON input. It accepts a spec file and an input file (or standard input).

Exit Codes:

• 0: Success
• 1: Error encountered

Usage: jolt transform [-h] [-u] spec [input]

Arguments:

• spec: Path to the JSON Jolt Transform Spec file.
• input: Path to the JSON input file. If omitted, the tool reads from stdin.

Options:

• -h, --help: Show help message.
• -u: Turns off pretty printing. Output will be raw JSON without formatting (default is false).

jolt transform spec.json input.json