GJSON: High-Performance JSON Retrieval for Go

repository·master·Indexed 12 days ago

https://github.com/tidwall/gjson

A high-performance Go package for fast and simple retrieval of values from JSON documents using dot notation and advanced query syntax. GJSON supports wildcards, array indexing, JSON Lines, and custom modifiers to transform data without requiring full unmarshaling into structs.

Tokens
7.3K
Snippets
37
Records
40
Agent score
92%

What's inside GJSON

  1. Understand Dot (.) vs Pipe (|) Separators

    master

    While . and | are often interchangeable, they behave differently when used after array/query operators (# or #( )):

    • Dot (.): Processes the path on each element of the preceding array before returning the results (mapping).
    • Pipe (|): Processes the path on the result of the previous operation as a single entity (chaining).

    If the previous result is an array, | treats that array as the new target, whereas . treats the elements inside the array as the targets.

    // If friends.#(last="Murphy")# returns an array of objects:
    
    friends.#(last="Murphy")#.first     // Returns an array of 'first' names: ["Dale", "Jane"]
    friends.#(last="Murphy")#|first     // Returns <non-existent> because it looks for 'first' on the array itself
    friends.#(last="Murphy")#|0         // Returns the first object in that array
  2. Work with JSON Arrays using #

    master

    The # character is used to interact with JSON arrays:

    • Use # alone to get the length of an array.
    • Use #.key to get an array of values for a specific key from all objects in the array.
    friends.#              // Returns the length of the array
    friends.#.age         // Returns an array of all 'age' values: [44, 68, 47]
  3. Construct New Documents with Multipaths

    master

    Multipaths allow you to join multiple selected values into a new JSON document. You can wrap comma-separated paths in [...] to create a new array or {...} to create a new object.

    • If a key is provided (e.g., "key":path), that key is used.
    • If no key is provided, the original field name is used.
    • If a name cannot be determined, _ is used.
    // Creates a new object with specific fields
    {name.first,age,"the_murphys":friends.#(last="Murphy")#.first}
    
    // Results in:
    // {"first":"Tom","age":37,"the_murphys":["Dale","Jane"]}
  4. Query Arrays with Expressions

    master

    You can filter arrays using queries inside #( ... ).

    Operators:

    • Comparison: ==, !=, <, <=, >, >=
    • Pattern Matching: % (like), !% (not like)
    • Boolean Conversion: ~ (tilde) converts a value to a boolean before comparison.

    Query Types:

    • #(...): Returns the first match.
    • #(...)#: Returns all matches.

    Tilde (~) Comparison Types:

    • ~true: Converts true-ish values to true.
    • ~false: Converts false-ish and non-existent values to true.
    • ~null: Converts null and non-existent values to true.
    • ~*: Converts any existing value to true.
    // Find first match
    friends.#(last=="Murphy").first     // "Dale"
    
    // Find all matches
    friends.#(last=="Murphy")#.first    // ["Dale","Jane"]
    
    // Pattern matching
    friends.#(first%"D*").last          // "Murphy"
    
    // Boolean conversion (true-ish)
    vals.#(b==~true)#.a                  // [2,6,7,8]
    
    // Existence check
    vals.#(b!=~*)#.a                      // [11]
  5. Use Wildcards in GJSON Paths

    master

    You can use wildcard characters to match keys within a JSON structure:

    • *: Matches any zero or more characters.
    • ?: Matches exactly one character.
    child*.2               // Matches any key starting with 'child' and returns its 3rd element
    c?ildren.0             // Matches 'children' with one character variation
  6. Use modifiers and path chaining

    master

    Modifiers perform custom processing on JSON and can be applied using the pipe | character. You can chain multiple modifiers together.

    Built-in Modifiers

    • @reverse: Reverses an array or object members.
    • @ugly: Removes all whitespace.
    • @pretty: Makes JSON human-readable.
    • @this: Returns the current element (can be used to get the root).
    • @valid: Ensures the document is valid.
    • @flatten: Flattens an array.
    • @join: Joins multiple objects into one.
    • @keys: Returns an array of keys.
    • @values: Returns an array of values.
    • @tostr: Converts JSON to a string.
    • @fromstr: Unwraps a JSON string.
    • @group: Groups arrays of objects.
    • @dig: Searches for a value without the full path.

    Modifier Arguments

    Modifiers can take arguments using a colon :. Arguments can be JSON or plain characters. Example: @pretty:{"sortKeys":true}

    Custom Modifiers

    Register custom logic using gjson.AddModifier(name, func(json, arg string) string).

    gjson.AddModifier("case", func(json, arg string) string {
      if arg == "upper" {
        return strings.ToUpper(json)
      }
      if arg == "lower" {
        return strings.ToLower(json)
      }
      return json
    })
  7. Path Syntax and Querying

    master

    GJSON uses a dot-separated path syntax to navigate JSON.

    Basic Navigation

    • Dot notation: name.last accesses nested keys.
    • Array index: children.1 accesses the second element of an array.
    • Wildcards: * and ? can be used for pattern matching.
    • Array length/Child access: Use # to get the number of elements in an array or to access a child path.
    • Escaping: Use \ to escape dots or wildcards in keys (e.g., fav\.movie).

    Advanced Queries

    Use #(...) to query an array:

    • First match: friends.#(last=="Murphy").first returns the first match.
    • All matches: friends.#(last=="Murphy")#.first returns all matching elements.

    Supported Operators:

    • Comparison: ==, !=, <, <=, >, >=
    • Pattern matching: % (like) and !% (not like)
  8. Use JSON Literals in Multipaths

    master

    Starting with v1.12.0, you can use JSON literals within multipaths to construct static blocks of JSON. A literal begins with the ! character.

    // Constructing a document with static values
    {name.first,age,"company":!"Happysoft","employed":!true}
    
    // Results in:
    // {"first":"Tom","age":37,"company":"Happysoft","employed":true}
  9. Get a value from JSON using dot notation

    master

    Use gjson.Get(json, path) to search a JSON document for a specific value. The path uses dot syntax (e.g., name.last or age). When the value is found, it is returned immediately as a gjson.Result.

    package main
    
    import "github.com/tidwall/gjson"
    
    const json = `{"name":{"first":"Janet","last":"Prichard"},"age":47}`
    
    func main() {
    	value := gjson.Get(json, "name.last")
    	println(value.String())
    }
  10. Escape Special Characters in Paths

    master

    Special characters like ., *, and ? can be escaped using a backslash \.

    Important: When hardcoding paths in source code, you must account for the language's own string escaping rules. In Go and Rust, you must escape the backslash itself if using standard double-quoted strings.

    // Go: must escape the slash for the string literal
    val := gjson.Get(json, "fav\\.movie") 
    
    // Go: no need to escape the slash when using raw string literals
    val := gjson.Get(json, `fav\.movie`) 
    // Rust: must escape the slash
    let val = gjson::get(json, "fav\\.movie")
    
    // Rust: no need to escape the slash with raw strings
    let val = gjson::get(json, r#"fav\.movie"#)
  11. Extract a zero-allocation sub-slice from JSON bytes

    master

    To avoid allocating a new byte slice when accessing result.Raw from a gjson.GetBytes call, you can use the result.Index field to create a sub-slice of the original JSON. This is a best-effort approach to achieve zero allocations. Note that if result.Index is 0, you must fall back to converting result.Raw to a []byte.

    var json []byte = ...
    result := gjson.GetBytes(json, path)
    var raw []byte
    if result.Index > 0 {
        raw = json[result.Index:result.Index+len(result.Raw)]
    } else {
        raw = []byte(result.Raw)
    }