json_repair

repository·main·Indexed 26 days ago

https://github.com/mangiucugna/json_repair

A lightweight Python library (v0.61.7) designed to repair malformed JSON data from LLMs, APIs, logs, and user input. It provides drop-in replacements for the standard library, including json_repair.loads(), load(), and from_file(). Key features include fixing syntax errors, noise removal, stream-stable repair, and schema-guided repairs using JSON Schema or Pydantic v2 models. It also includes a CLI for repairing files or stdin.

Tokens
4.8K
Snippets
20
Records
35
Agent score
88%

What's inside json_repair

  1. Supported JSON repair use cases

    main

    The json_repair library is designed to handle several types of malformed JSON input:

    • Syntax Errors: Missing quotes, comma errors, unescaped characters, incomplete key-value pairs, and incorrect boolean/null values (e.g., fixing true/false/null).
    • Broken Arrays and Objects: Completing unfinished arrays or objects by adding necessary elements (commas, brackets) or default values (null, "").
    • Noise Removal: Cleaning strings that contain extra non-JSON characters like comments or miscellaneous text while preserving the structure.
    • Auto-completion: Automatically providing reasonable default values (like empty strings or null) for missing fields to ensure the JSON remains parsable.
  2. Perform Schema-guided JSON repair

    main

    You can use a JSON Schema or a Pydantic v2 model to guide the repair process. This allows the parser to fill in missing values, perform safe type conversions (e.g., "1" $\rightarrow$ 1), and remove disallowed fields.

    Modes (schema_repair_mode):

    • standard (default): Standard schema-guided behavior.
    • salvage: Attempts to return usable arrays/objects even if some items are unrepairable (e.g., dropping unrepairable array items).

    Note: Schema-guided repair is mutually exclusive with strict=True.

    Using a JSON Schema:

    from json_repair import repair_json
    
    schema = {
        "type": "object",
        "properties": {"value": {"type": "integer"}},
        "required": ["value"],
    }
    
    repair_json('{"value": "1"}', schema=schema, return_objects=True)

    Using a Pydantic v2 Model:

    from pydantic import BaseModel, Field
    from json_repair import repair_json
    
    class Payload(BaseModel):
        value: int
        tags: list[str] = Field(default_factory=list)
    
    repair_json(
        '{"value": "1", "tags": }',
        schema=Payload,
        skip_json_loads=True,
        return_objects=True,
    )
  3. Optimize performance with skip_json_loads and return_objects

    main

    To improve performance when you are 100% certain the input is invalid JSON, use the following flags:

    • skip_json_loads=True: Skips the initial strict validation using the standard library, going directly to the repair parser.
    • return_objects=True: Returns the parsed object directly, which is faster than returning a string.

    Warning: Do not use skip_json_loads=True for valid JSON, as the repair parser might alter the structure or values.

    from json_repair import repair_json
    
    obj = repair_json(bad_json_string, return_objects=True, skip_json_loads=True)
  4. Repair malformed JSON with json_repair.loads()

    main

    Use json_repair.loads() as a drop-in replacement for json.loads() to fix common JSON issues such as missing quotes, misplaced commas, unescaped characters, incomplete key-value pairs, and truncated values. It can also handle JSON containing comments or stray prose.

    import json_repair
    
    bad_json = '{"users":[{"name":"Ada","role":"admin",}],"ok":true'
    decoded_object = json_repair.loads(bad_json)
    
    # Result: {'users': [{'name': 'Ada', 'role': 'admin'}], 'ok': True}
  5. Replace json.loads() with json_repair.loads()

    main

    You can use json_repair.loads() as a drop-in replacement for the standard library json.loads(). It automatically attempts to repair the JSON before parsing. This is the recommended way to handle potentially malformed JSON.

    import json_repair
    
    decoded_object = json_repair.loads(json_string)
  6. Enable strict mode for JSON validation

    main

    By default, the library attempts to repair errors. If you want to enforce strict validation and raise a ValueError on structural issues (like duplicate keys, missing colons, or empty keys/values), use strict=True.

    from json_repair import repair_json
    
    repair_json(bad_json_string, strict=True)
  7. Use json_repair.loads() as a drop-in replacement for json.loads()

    main

    json_repair.loads() is the recommended way to parse JSON. It first attempts a strict parse using the standard library json.loads(). If that fails, it automatically switches to the repair parser.

    import json_repair
    
    decoded_object = json_repair.loads(json_string)
  8. Perform Schema-guided repairs

    main

    You can guide repairs using a JSON Schema or a Pydantic v2 model. This allows the parser to fill missing values, coerce scalars, and drop disallowed properties.

    Note: This requires the [schema] extra: pip install 'json-repair[schema]'. Schema guidance is mutually exclusive with strict=True.

    Available schema_repair_mode values:

    • standard (default): Basic schema guidance.
    • salvage: Best-effort salvage of arrays/objects (e.g., dropping unrepairable array items or unwrapping single-item arrays).
    from pydantic import BaseModel, Field
    from json_repair import repair_json
    
    class Payload(BaseModel):
        value: int
        tags: list[str] = Field(default_factory=list)
    
    # Using a Pydantic model as a schema
    repair_json(
        '{"value": "1", "tags": }',
        schema=Payload,
        skip_json_loads=True,
        return_objects=True,
    )