JSONSchemer

repository·main·Indexed 19 days ago

https://github.com/davishmcclurg/json_schemer

A Ruby JSON Schema validator supporting multiple drafts (4, 6, 7, 2019-09, 2020-12) and OpenAPI specifications (3.0, 3.1). It includes support for custom error messages via the x-error keyword with variable interpolation for instance and keyword locations.

Tokens
7.9K
Snippets
28
Records
41
Agent score
66%

What's inside json_schemer

  1. Overview of JSON Schema Test Suite

    main

    The JSON Schema Test Suite is a language-agnostic collection of JSON objects designed to test JSON Schema validation libraries. It provides test cases for multiple specification releases, including draft-2020-12, draft-2019-09, draft-07, draft-06, draft-04, and draft-03.

    To use this suite in your project, it is recommended to clone the main branch as a git submodule or git subtree to ensure you are using a stable version.

  2. Format of Output Test files

    main

    Output test files follow a structure similar to standard validation tests, but with the valid property removed and replaced by an output property.

    The output property contains schemas for each supported output format. A compliant implementation's output must validate against these schemas.

    Example structure concept:

    {
      "output": {
        "basic": { "$ref": "..." },
        "detailed": { "$ref": "..." }
      }
    }
  3. How to handle expected annotations and multiple contributors

    main

    In an assertion, the expected field is an object where keys are schema locations and values are the annotations contributed by those locations.

    Because multiple schema locations can contribute to a single keyword annotation, the expected object may contain multiple entries.

    Convention: For the purpose of this suite, the expected entries should be sorted such that the most recently encountered value (based on a top-down evaluation of the schema) appears before previously encountered values.

  4. Customize error messages using the `x-error` keyword

    main

    You can override default JSON Schema error messages by adding the x-error keyword to your schema. x-error takes precedence over I18n translations.

    It supports several modes:

    • String: A single string used for the schema and all its keywords.
    • Hash: Allows keyword-specific overrides.
      • Use the specific keyword name (e.g., 'type') for targeted messages.
      • Use '^' to define a fallback error for the schema itself.
      • Use '*' to define a fallback error for the schema and all its keywords.

    Variable Interpolation Inside an x-error string or template, you can use the following variables to inject context into the error message:

    • %{instance}: The data being validated.
    • %{instanceLocation}: The JSON pointer to the data.
    • %{formattedInstanceLocation}: The JSON pointer formatted as a code literal.
    • %{keywordValue}: The value of the keyword that failed.
    • %{keywordLocation}: The JSON pointer to the keyword in the schema.
    • %{absoluteKeywordLocation}: The absolute URI to the keyword in the schema.
    • %{details}: A hash of additional error details.
    • %{details__<key>}: Accesses a specific key within the details hash (e.g., %{details__missing_keys}).
    # keyword-specific errors
    schemer = JSONSchemer.schema({
      'type' => 'string',
      'minLength' => 10,
      'x-error' => {
        'type' => 'custom error for `type` keyword',
        '^' => 'custom error for schema',
        '*' => 'fallback error for schema and all keywords'
      }
    })
    
    # variable interpolation example
    schemer = JSONSchemer.schema({
      'properties' => {
        'abc' => {
          'type' => 'object',
          'required' => ['xyz'],
          'x-error' => "instance: %{instance}, location: %{instanceLocation}"
        }
      }
    })
  5. Understand JSON Schema Test Suite terminology

    main

    The suite uses specific terminology to describe its structure:

    TermDefinition
    test suiteThe entire repository containing tests for multiple JSON Schema specification releases.
    test caseA single JSON object containing a description, a schema, and an array of tests.
    testAn individual entry within a test case's tests array, containing a description, the data (instance) to be validated, and a valid boolean indicating the expected result.
    test runnerAn external program (authored by the user) that executes the tests in the suite against a validator implementation.
  6. Understand the limitations of the JSON Schema Test Suite

    main

    The test suite expresses assertions about implementation behavior strictly within the bounds of JSON Schema itself. Each test consists of applying a specific schema to a particular instance.

    Consequently, the suite can only test behaviors that a schema is capable of representing. It cannot test behaviors mandated by the JSON Schema specification that cannot be expressed via core vocabularies. For example, while the specification may contain recommendations regarding URI normalization, a JSON schema cannot currently represent an assertion for URI normalization within its core vocabularies; therefore, no tests in this suite cover that specific behavior.

  7. Understand the organization of Output Tests

    main

    Tests are organized by specification release and categorized into two types:

    1. Content tests: Verify that keywords produce correct annotations and/or error messages. They focus on the behavior of each keyword.

      • For 2019-09 and 2020-12 releases, these require the basic output format.
      • For later versions, these require the list output format.
    2. Structure tests: Verify that the output formats themselves (e.g., flag, basic, detailed, verbose, list, hierarchical) are structured correctly according to the spec. These do not need to cover every keyword, but must cover the various ways output structures are built.

    Each release folder contains an output-schema.json file which defines the expected output structure for that specific release. This schema must be loaded for the tests to function.

  8. Customize error messages using I18n

    main

    If the i18n gem is loaded, json_schemer looks up error messages under the json_schemer key. This is useful for multi-language support.

    Lookup Order Translations are looked up in the following order (the first match wins):

    1. $LOCALE.json_schemer.errors.$ABSOLUTE_KEYWORD_LOCATION
    2. $LOCALE.json_schemer.errors.$SCHEMA_ID.$KEYWORD_LOCATION
    3. $LOCALE.json_schemer.errors.$KEYWORD_LOCATION
    4. $LOCALE.json_schemer.errors.$SCHEMA_ID.$KEYWORD
    5. $LOCALE.json_schemer.errors.$SCHEMA_ID.*
    6. $LOCALE.json_schemer.errors.$META_SCHEMA_ID.$KEYWORD
    7. $LOCALE.json_schemer.errors.$META_SCHEMA_ID.*
    8. $LOCALE.json_schemer.errors.$KEYWORD
    9. $LOCALE.json_schemer.errors.*

    Note: You may need to restart your application after adding the root key because the existence check is cached for performance.

    en:
      json_schemer:
        errors:
          '#/properties/abc/required': custom error for keyword location
          'required': custom error for `required` keyword
          '^': custom error for schema
          '*': fallback error for schema and all keywords
  9. Handling ambiguity in 2019-09/2020-12 `basic` output

    main

    The 2019-09 and 2020-12 specifications have ambiguity regarding the basic output structure when only a single output node exists. There are two possible interpretations:

    1. Nested structure: The root schema appears in a list with a containing node that only has a valid property.
      {
          "valid": false,
          "errors": [
              {
                  "valid": false,
                  "keywordLocation": "",
                  "absoluteKeywordLocation": "...",
                  "instanceLocation": ""
              }
          ]
      }
    2. Collapsed structure: The entire structure collapses to just the root output node (similar to detailed).
      {
          "valid": false,
          "keywordLocation": "",
          "absoluteKeywordLocation": "...",
          "instanceLocation": ""
      }

    Workaround for test authors: To avoid testing for a specific interpretation, you can force a second output unit by adding an "anyOf": [ true ] to the schema. This adds enough structure to avoid the ambiguity without changing the validation result. Ensure your test schema targets the specific requirement and ignores the extra output units generated by this keyword.

    "anyOf": [ true ]
  10. Best practices for writing Output Test cases

    main

    When contributing new test cases to the suite, follow these guidelines:

    • Targeted Validation: Keep output validation schemas focused on verifying a single requirement.
    • Granularity: Where possible, create multiple small tests to cover multiple requirements rather than one large test. This improves readability and increases test coverage.
    • General Tests: For tests that do not pertain to a specific keyword, use the general.json file located within the content tests directory.