partial-json

repository·main·Indexed 42 days ago

https://github.com/promplate/partial-json-parser-js

A lightweight JavaScript library for parsing partial or incomplete JSON strings, specifically designed for streaming data from Large Language Models (LLMs). It provides a `parse()` function and an `Allow` enum bitmask to control which data types (such as objects, arrays, or strings) can be returned in a partial state before the final token is received. Version 0.1.7.

Tokens
2.6K
Snippets
10
Records
10
Agent score
54%

What's inside partial-json

  1. Configure allowed partial types using the Allow object

    main

    The Allow object is an Enum used to control which types of data can be returned in a partial state. You pass these constants to the parse function using a bitwise OR (|).

    If a type is not included in the allow bitmask, the parser will only include that value in the result once it is fully completed. For example, if you only allow OBJ but not STR, an incomplete string inside an object will cause that key to be omitted entirely until the string is closed.

    import { parse, OBJ, STR } from "partial-json";
    
    // Without STR, the key is omitted because the value is incomplete
    const result1 = parse('{"key": "v', OBJ);
    console.log(result1); // Outputs: {}
    
    // With STR, the partial value is captured
    const result2 = parse('{"key": "v', STR | OBJ);
    console.log(result2); // Outputs: { key: 'v' }
  2. Handle malformed JSON errors

    main

    If the input string is truly malformed (not just partial, but syntactically invalid in a way the parser cannot recover from), parse will throw a MalformedJSON [Error] containing a SyntaxError message.

    // This will throw an error
    parse("wrong"); 
    // Error: MalformedJSON [Error]: SyntaxError: Unexpected token 'w', "wrong" is not valid JSON at position 0
  3. Parse partial JSON strings with parse()

    main

    The parse function allows you to decode JSON strings that are incomplete (e.g., during LLM streaming). When a JSON string is incomplete, parse can return the data it has successfully gathered so far, provided you specify which types of partial data are allowed using the Allow object constants.

    import { parse, STR, OBJ } from "partial-json";
    
    // Returns partial data if STR and OBJ are allowed
    const result = parse('{"key": "v', STR | OBJ);
    console.log(result); // Outputs: { key: 'v' }
  4. Configure partial parsing behavior with Allow constants

    main

    The Allow object provides bitwise constants used to control which JSON types are permitted to be partially parsed. By default, the parser allows all types, but you can restrict behavior to prevent issues like gradually increasing number sizes or incomplete strings.

    Usage Patterns

    • Restrict to specific types: Pass a specific constant (or a bitwise combination) to the parse function to only allow those types to be partially parsed.
    • Disallow specific types: Use the bitwise NOT operator (~) to exclude a specific type from the default 'allow all' behavior.

    Available Constants

    ConstantDescription
    STRAllows partial strings (e.g., "hello \u12 becomes "hello ")
    NUMAllows partial numbers (e.g., 123. becomes 123)
    ARRAllows partial arrays (e.g., [1, 2, becomes [1, 2])
    OBJAllows partial objects (e.g., {"a": 1, "b": becomes {"a": 1})
    NULLAllows nu to be parsed as null
    BOOLAllows tr as true and fa as false
    NANAllows Na as NaN
    INFAllows Infinity or -Infinity
    SPECIALGrouping for NULL, BOOL, INF, and NAN
    ATOMGrouping for STR, NUM, and SPECIAL
    COLLECTIONGrouping for ARR and OBJ
    ALLAll types
    import { Allow, parse } from "partial-json";
    
    // Only allow arrays to be partially parsed
    parse(`[{"a": 1, "b": 2}, {"a": 3,`, Allow.ARR); // [ { a: 1, b: 2 } ]
    
    // Disallow partial objects using the bitwise NOT operator
    parse(`[{"a": 1, "b": 2}, {"a": 3,`, ~Allow.OBJ); // [ { a: 1, b: 2 } ]
    
    // Disallow partial strings
    parse(`["complete string", "incompl`, ~Allow.STR); // [ 'complete string' ]
  5. Reference: Allow constants

    main

    The Allow object contains bitwise constants that define which partial types are acceptable during parsing. You can import these directly from partial-json.

    // Available constants in the Allow object:
    // STR: Allow partial string.
    // NUM: Allow partial number.
    // ARR: Allow partial array.
    // OBJ: Allow partial object.
    // NULL: Allow partial null.
    // BOOL: Allow partial boolean.
    // NAN: Allow partial NaN.
    // INFINITY: Allow partial Infinity.
    // _INFINITY: Allow partial -Infinity.
    // INF: Allow both partial Infinity and -Infinity.
    // SPECIAL: Allow all special values.
    // ATOM: Allow all atomic values.
    // COLLECTION: Allow all collection values.
    // ALL: Allow all values (default).
  6. Parse partial or complete JSON with parse()

    main

    The parse function is the primary entrypoint for parsing JSON strings that may be incomplete or malformed. It allows you to specify which types of data are permitted to be returned in a partial state using the Allow bitmask.

    Parameters

    • jsonString (string): The JSON string to be parsed.
    • allowPartial (number, optional): A bitmask from the Allow enum specifying which types are allowed to be partial. Defaults to Allow.ALL.

    Returns

    • The parsed JSON object, array, string, number, boolean, or null.

    Errors

    • Throws PartialJSON if the JSON is incomplete based on the allow configuration.
    • Throws MalformedJSON if the JSON is syntactically invalid and cannot be recovered.
    • Throws TypeError if the input is not a string.
    • Throws Error if the input string is empty.
    import { parse, Allow } from 'partial-json';
    
    // Parse a complete JSON string
    const complete = parse('{"key": "value"}');
    
    // Parse a partial JSON string by allowing partial objects
    const partial = parse('{"key": "value"', Allow.OBJ);
  7. Handle partial and malformed JSON errors

    main

    When using parse or parseJSON, you should handle two specific error classes to distinguish between incomplete data and invalid syntax:

    • PartialJSON: Thrown when the input ends prematurely but the allow parameter permits the current state (e.g., an unterminated string or object).
    • MalformedJSON: Thrown when the input contains syntax that is fundamentally invalid and cannot be parsed even as a partial structure.

    Both errors include the error message and the character position where the issue was detected.

    import { parse, PartialJSON, MalformedJSON } from 'partial-json';
    
    try {
      const data = parse('{"key": "val');
    } catch (e) {
      if (e instanceof PartialJSON) {
        console.error('JSON is incomplete:', e.message);
      } else if (e instanceof MalformedJSON) {
        console.error('JSON is invalid:', e.message);
      }
    }
  8. Configure partial parsing with the Allow enum

    main

    The Allow enum is a bitmask used to control which JSON types are allowed to be returned in a partial/incomplete state. By default, parse uses Allow.ALL.

    To restrict partial parsing, pass a combination of these flags to the allowPartial argument of parse():

    • Allow.ALL: Allows all types to be partial.
    • Allow.OBJ: Allows partial objects (e.g., {"a": 1 returns {"a": 1}).
    • Allow.ARR: Allows partial arrays (e.g., [1, 2 returns [1, 2]).
    • Allow.STR: Allows partial strings (e.g., "hello returns "hello").
    • Allow.NUM: Allows partial numbers.
    • Allow.BOOL: Allows partial booleans.
    • Allow.NULL: Allows partial null values.
    • Allow.NAN: Allows partial NaN.
    • Allow.INFINITY: Allows partial Infinity.
    • Allow._INFINITY: Allows partial -Infinity.
    // Example: Only allow partial objects and arrays
    import { parse, Allow }
    
    const result = parse('[1, {"a": 1', Allow.ARR | Allow.OBJ);
  9. Reference: parse() function signature

    main

    The primary function for decoding JSON strings.

    /**
     * @param jsonString <string> - The JSON string to parse.
     * @param allowPartial <number> - Specify what kind of partialness is allowed (default: Allow.ALL).
     * @returns The parsed JavaScript value.
     */
    function parse(jsonString: string, allowPartial?: number): any;