jackson-jq

repository·develop/2.x·Indexed 43 days ago

https://github.com/eiiches/jackson-jq

A pure-Java, embeddable implementation of the jq JSON processor. It features a pluggable architecture supporting multiple JSON providers including Jackson 2, Jackson 3, Gson, and Jakarta. Version 2.0.0-alpha2 provides a CLI for testing and emulating specific jq versions, along with a comprehensive set of implemented jq functions and operators.

Tokens
7.4K
Snippets
23
Records
31
Agent score
47%

What's inside jackson-jq

  1. Understand jackson-jq compatibility with jq

    develop/2.x

    While jackson-jq aims for compatibility with jq, it is not a 1:1 implementation. Some jq features are omitted because they are not relevant to a Java library, while others may be incomplete or contain bugs.

    When writing filters, be aware that not every feature from the jq 1.5 manual is guaranteed to be available in jackson-jq. Always verify feature availability if your logic relies on advanced jq functionality.

  2. How extension modules work in jackson-jq

    develop/2.x

    Extension modules provide additional functions not found in standard jq. There are two types of modules:

    1. JavaModule: Functions written in Java that are ready to call.
    2. JqModule: jq source code that is loaded and compiled by the module loader.

    To use extension modules in a Java application, you must add the required dependencies and configure an Environment with a ClassPathModuleLoader.

    • EnvironmentBuilder.withDefaultLoaders(...) installs a default loader that searches the classpath and the library's own class loader.
    • addModuleLoader(...) appends a new loader to the search order.
    • clearModuleLoaders() removes default loaders if you want to define a completely custom search order.

    Once the dependency is added, you must explicitly import the module in your jq program using its namespace. Extension functions are only accessible through their imported namespace.

    import "jackson-jq/uuid" as uuid;
    
    uuid::uuid4
  3. Install jackson-jq via Maven

    develop/2.x

    To use jackson-jq in your Java project, you must include jackson-jq-core and exactly one JSON provider dependency in your pom.xml.

    Requirements:

    • Java 8 or later.
    • If your queries use regex functions, you must also include the jackson-jq-regex-impl-joni dependency.

    JSON Provider Options:

    • jackson-jq-json-provider-impl-jackson2: Standard Jackson 2 provider.
    • jackson-jq-json-provider-impl-jackson3: Requires Java 17.
    • jackson-jq-json-provider-impl-gson: Uses Google Gson.
    • jackson-jq-json-provider-impl-jakarta: Requires Java 11 and a JSON-P 2.1 implementation (e.g., org.eclipse.parsson:parsson).
    <dependency>
    	<groupId>net.thisptr.jackson.jq.v2</groupId>
    	<artifactId>jackson-jq-core</artifactId>
    	<version>2.0.0-alpha2</version>
    </dependency>
    
    <!-- Optional: add this dependency if your application uses regex functions -->
    <dependency>
    	<groupId>net.thisptr.jackson.jq.v2</groupId>
    	<artifactId>jackson-jq-regex-impl-joni</artifactId>
    	<version>2.0.0-alpha2</version>
    	<scope>runtime</scope>
    </dependency>
    
    <!-- Choose one JSON provider -->
    <dependency>
    	<groupId>net.thisptr.jackson.jq.v2</groupId>
    	<artifactId>jackson-jq-json-provider-impl-jackson2</artifactId>
    	<version>2.0.0-alpha2</version>
    </dependency>
  4. Enable Regex functions via jackson-jq-regex-impl-joni

    develop/2.x

    By default, jackson-jq does not include regex support. To use regex functions such as test, match, capture, scan, sub, gsub, splits, and split/2, you must add the jackson-jq-regex-impl-joni module to your project. This module is backed by the Joni regex engine.

    In a Java project using Maven, add the following dependency:

    Functions are discovered automatically via ServiceLoader, so no manual registration is required in your code. If you are using the jackson-jq command-line application, this module is included by default.

    <dependency>
    	<groupId>net.thisptr.jackson.jq.v2</groupId>
    	<artifactId>jackson-jq-regex-impl-joni</artifactId>
    	<version>2.0.0-alpha2</version>
    </dependency>
  5. Run jackson-jq benchmarks

    develop/2.x

    Benchmarks are performed using JMH and measure query compilation and application separately. By default, the benchmark uses the jackson3 provider and jq 1.8.2 compatibility semantics.

    Usage Pattern: bazelisk run //jackson-jq-benchmark -- [options] <jq_expression> <json_input> [jmh_options]

    Supported JSON Providers:

    • jackson2
    • jackson3
    • fastjson2
    • gson
    • jakarta

    Supported jq Versions:

    • Specified via --jq-version (e.g., 1.7).
    # Default benchmark run
    bazelisk run //jackson-jq-benchmark -- '.items | map(.price) | add' '{"items":[{"price":10},{"price":20}]}'
    
    # Benchmark with custom provider and jq version
    bazelisk run //jackson-jq-benchmark -- \
    	--json-provider gson \
    	--jq-version 1.7 \
    	'.foo' \
    	'{"foo":42}' \
    	-f 3
  6. Implement the env/0 function using EnvironmentBuilder

    develop/2.x

    For security reasons, jackson-jq does not provide a built-in implementation of env/0. If your jq scripts require access to environment variables, you must implement the function yourself and register it with the EnvironmentBuilder.

    builder.defineFunction(FunctionSignature.of("env", 0), new Function() {
    	@Override
    	public <Context extends RuntimeContext, T> Expression<Context, T> bind(BindContext<T> bindCtx, List<Expression<Context, T>> args) {
    		JsonProvider<T> jsonProvider = bindCtx.getJsonProvider();
    		return (context, in, ipath, output) -> {
    			Map<String, T> result = new HashMap<>();
    			for (Map.Entry<String, String> entry : System.getenv().entrySet()) {
    				result.put(entry.getKey(), jsonProvider.createString(entry.getValue()));
    			}
    			output.emit(jsonProvider.createObject(result), UntrackedPath.getInstance());
    		};
    	}
    });
  7. Test Maven consumer projects (smoke-tests)

    develop/2.x

    The Maven projects located in smoke-tests/ are consumers of the library. To test them, you must first publish the Bazel-built artifacts to your local Maven repository using the //:maven_install target.

    bazel run //:maven_install
    for project in java8 jpms osgi quarkus graalvm; do
    	mvn -f "smoke-tests/$project/pom.xml" clean verify
    done
  8. Build jackson-jq from source

    develop/2.x

    The project uses Bazel 9. It is recommended to use bazelisk to ensure the correct version is used as pinned in .bazelversion.

    Before submitting changes, you should format the code, check Bazel file linting, and run the full test suite:

    1. Check code formatting: bazelisk run //:idea-format -- --check
    2. Check Bazel file linting: bazel run //:buildifier_test
    3. Run all tests: bazel test //...
    bazelisk run //:idea-format -- --check
    bazel run //:buildifier_test
    bazel test //...
  9. Use _modify/2 to replicate jq behavior for '.|= empty'

    develop/2.x

    The expression .foo |= empty is considered undefined in jq and results in an error in jackson-jq. If you need to replicate the specific (and often incorrect) behaviors found in jq 1.5 or jq 1.6, use the _modify/2 function directly.

    # To replicate jq 1.6 behavior:
    $ java -jar jackson-jq-cli-2.0.0-alpha2.jar --jq 1.6 -n '[1,2,3] | _modify((.[] | select(. > 1)); empty)'
    # Output: [ 1, 3 ]
    
    # To replicate jq 1.5 behavior:
    $ java -jar jackson-jq-cli-2.0.0-alpha2.jar --jq 1.5 -n '[1,2,3] | _modify((.[] | select(. > 1)); empty)'
    # Output: null
  10. How jq input options affect data processing

    develop/2.x

    The jackson-jq CLI processes input based on specific flags that change how data is read and structured. These options determine whether the input is treated as JSON, raw text, or a single concatenated stream.

    Input Modes

    FlagNameBehavior
    -nnullInputFeeds a single null value and reads nothing. This takes precedence over all other options.
    -RrawInputTreats input as raw text. If used without -s, it passes each line (separated by \n) as a string. If used with -s, it reads the entire input as a single string.
    -sslurpCollects the entire run of input files/values into a single array. When used with JSON input, it wraps all parsed values into one array.
    (default)JSONParses the input as a stream of JSON values. Multiple JSON values in the input are processed sequentially.

    Line Termination Behavior

    When using raw input (-R), lines are separated strictly by the \n character. A \r (part of a CRLF terminator) is preserved as part of the line content. Files are treated as one continuous concatenated input; a partial line at the end of one file is carried over to the start of the next.

  11. How jq input options affect data flow

    develop/2.x

    In jackson-jq, input options like -n, -R, and -s do not change the execution logic of a query itself. Instead, they change the nature of the InputSource—the sequence of values that the query is applied to.

    Depending on the flags used, the query receives different types of input:

    • A stream of parsed JSON values.
    • One string per input line (raw input).
    • A single array containing all input data (slurp mode).

    Each of these modes is implemented as an InputSource, which manages how data is read and passed to the query engine.

  12. Use the jackson-jq CLI

    develop/2.x

    The jackson-jq command-line interface allows you to run jq filters on JSON data. The basic usage pattern is:

    jackson-jq [OPTIONS...] QUERY [FILE...]

    If you use the --from-file option, the first positional argument is treated as the filter file, and all subsequent arguments are treated as input files. Otherwise, the first positional argument is the query string, and the rest are input files. If no input files are provided, it reads from standard input.

    jackson-jq [OPTIONS...] QUERY [FILE...]