GenSON Documentation

repository·master·Indexed 20 days ago

https://github.com/wolverdude/genson

GenSON is a Python-based JSON Schema generator that automatically infers structure from data by merging multiple JSON objects and existing schemas into a single, unified schema. It features a customizable SchemaBuilder using the Strategy Pattern, allowing developers to implement custom SchemaStrategy classes to track specific keywords like minimum, maximum, or to modify how required fields and enums are handled.

Tokens
6.4K
Snippets
15
Records
21
Agent score
71%

What's inside GenSON

  1. How SchemaBuilder objects interact

    master

    You can interact with SchemaBuilder instances in two primary ways:

    1. Merging: You can pass one SchemaBuilder instance directly into the add_schema() method of another to merge their schemas.
    2. Comparison: You can use the == operator to check if two SchemaBuilder objects have resulted in identical schemas.
    from genson import SchemaBuilder
    
    b1 = SchemaBuilder()
    b1.add_schema({"type": "object", "properties": {"hi": {"type": "string"}}})
    
    b2 = SchemaBuilder()
    b2.add_schema({"type": "object", "properties": {"hi": {"type": "integer"}}})
    
    # Comparison returns False initially
    print(b1 == b2) # False
    
    # Merging b2 into b1
    b1.add_schema(b2)
    
    # Comparison returns True after merge
    print(b1 == b2) # True
  2. Force the 'required' keyword in output

    master

    GenSON automatically manages the required array by including keys that appear in every object processed. If the intersection of keys becomes empty, GenSON drops the required key entirely.

    To force an empty required list to appear in the output (useful when merging builders to prevent divergence), seed the builder with: {"type": "object", "required": []}.

    # To ensure 'required' is present even if empty
    builder.add_schema({'type': 'object', 'required': []})
  3. Enable and use enums via seeding

    master

    GenSON does not infer enum keywords automatically. To use enums, you must activate them by seeding a node with a schema containing the enum keyword (an empty list [] is sufficient to activate it).

    Behavioral notes:

    • Once activated, the node captures all encountered values into a deduplicated list instead of inferring a type.
    • Supported values: Only scalar values (string, number, boolean, and null) are supported. Passing lists or objects to an enum node will raise a TypeError.
    • Ordering: Values are merged as a set; the order in the output list is not guaranteed.
    • Const: To simulate a const keyword, seed with enum, and if only one value is captured, manually replace the enum block with {"const": value}.
    >>> from genson import SchemaBuilder
    >>> builder = SchemaBuilder()
    >>> # Activate enum for the 'status' property
    >>> builder.add_schema({'type': 'object', 'properties': {
    ...     'status': {'enum': []}
    ... }})
    >>> builder.add_object({'status': 'active'})
    >>> builder.add_object({'status': 'inactive'})
    >>> builder.add_object({'status': 'active'})
    >>> builder.to_schema()
    {'$schema': 'http://json-schema.org/schema#', 
     'type': 'object', 
     'properties': {'status': {'enum': ['inactive', 'active']}}, 
     'required': ['status']}
  4. Understand typeless schemas in GenSON

    master

    GenSON supports typeless schemas (schemas without a type keyword) to facilitate flexible seeding.

    Warning: GenSON incorporates typeless schemas into the first-available typed schema. Because typed schemas are stricter than typeless ones, an object that would have validated under the original typeless schema might not validate under the resulting merged schema.

  5. Use seed schemas to control array validation (List vs Tuple)

    master

    By default, GenSON interprets arrays using List validation (one schema for all items). To force Tuple validation (a different schema for every array position), seed the SchemaBuilder with a schema containing an items array.

    Note: You can provide an 'imperfect' seed schema (e.g., an empty items list); GenSON will automatically adjust it to be a valid JSON Schema.

    >>> from genson import SchemaBuilder
    >>> builder = SchemaBuilder()
    >>> # Default behavior: List validation
    >>> builder.add_object(['one', 1])
    >>> builder.to_schema()
    {'$schema': 'http://json-schema.org/schema#', 'type': 'array', 'items': {'type': ['integer', 'string']}}
    
    >>> # Forced behavior: Tuple validation via seeding
    >>> builder = SchemaBuilder()
    >>> seed_schema = {'type': 'array', 'items': []}
    >>> builder.add_schema(seed_schema)
    >>> builder.add_object(['one', 1])
    >>> builder.to_schema()
    {'$schema': 'http://json-schema.org/schema#', 'type': 'array', 'items': [{'type': 'string'}, {'type': 'integer'}]}
  6. How GenSON releases are published

    master

    GenSON releases are automated via GitHub Actions using PyPI trusted publishing. Pushing a Git tag triggers the Publish workflow.

    Crucial Note: The version published to PyPI is determined by the __version__ string in genson/__init__.py, not by the Git tag name. The Git tag merely triggers the workflow. If the __version__ in the code and the Git tag do not match, the workflow will fail.

    Version strings must follow PEP 440. For release candidates, use the format 1.4.0rc1, 1.4.0rc2, etc. Do not use a fourth dotted number (like 1.4.0.1) for test releases, as these sort incorrectly in package resolvers.

  7. Use seed schemas to enable patternProperties

    master

    GenSON defaults to using properties. To use patternProperties (validating keys against RegEx strings), you must seed the builder with an object schema containing a patternProperties dictionary.

    Important details:

    • Uses Python-flavor RegEx.
    • If a property matches both a specific properties entry and a patternProperties regex, GenSON prefers and updates the properties entry.
    • If a key matches multiple patterns, the specific pattern updated is not guaranteed.
    • You can use None as a placeholder value for the pattern keys in your seed schema.
    >>> from genson import SchemaBuilder
    >>> builder = SchemaBuilder()
    >>> # Seed with patternProperties using None as a placeholder
    >>> builder.add_schema({'type': 'object', 'patternProperties': {r'^\d+$': None}})
    >>> builder.add_object({'1': 1, '2': 2, '3': 3})
    >>> builder.to_schema()
    {'$schema': 'http://json-schema.org/schema#', 'type': 'object', 'patternProperties':  {'^\\d+$': {'type': 'integer'}}}
  8. Perform a final release to PyPI

    master

    Once the dry run on TestPyPI is successful, publish the final version to PyPI:

    1. Set __version__ in genson/__init__.py to the final version (e.g., '1.4.0').
    2. Commit and push the change.
    3. Tag the commit and push the tag.

    The workflow will publish to PyPI and run the verify-pypi job. Finally, check the PyPI project page to ensure documentation renders correctly.

    # 1. Update genson/__init__.py: __version__ = '1.4.0'
    
    # 2. Commit and push
    git commit -am "Release 1.4.0" && git push
    
    # 3. Tag and push
    git tag v1.4.0 && git push origin v1.4.0
  9. How to customize SchemaBuilder

    master

    To add custom logic (such as tracking specific JSON schema keywords like minimum or maximum), you must follow a three-step process:

    1. Create a custom SchemaStrategy class: Inherit from an existing strategy (like SchemaStrategy or TypedSchemaStrategy) and override methods to handle new keywords or data types.
    2. Create a SchemaBuilder subclass: Incorporate your custom strategy into a new builder class.
    3. Use your custom SchemaBuilder: Use this subclass instead of the standard SchemaBuilder to perform schema generation.

    GenSON uses the Strategy Pattern to map nodes in an object or schema to specific strategy instances. Each instance maintains the state for that specific node.

    from genson import SchemaBuilder
    from genson.schema.strategies import Number
    
    class MinNumber(Number):
        # 1. Custom Strategy
        KEYWORDS = (*Number.KEYWORDS, 'minimum')
        # ... implementation ...
    
    class MinNumberSchemaBuilder(SchemaBuilder):
        # 2. Custom Builder
        EXTRA_STRATEGIES = (MinNumber,)
    
    # 3. Usage
    builder = MinNumberSchemaBuilder()
    builder.add_object(5)
    print(builder.to_schema())
  10. Dry run a release against TestPyPI

    master

    Before a final release, perform a dry run using a release candidate (rc) version to verify the packaging mechanics. This process uses TestPyPI to ensure the package can be installed, imported, and run via CLI without errors.

    1. Set __version__ in genson/__init__.py to the rc version (e.g., '1.4.0rc1').
    2. Commit and push the change.
    3. Tag the commit and push the tag.

    If the verify-testpypi GitHub Action job passes, the mechanics are correct. If you need to try again, you must increment the rc number (e.g., rc2) because TestPyPI does not allow re-uploading the same version.

    # 1. Update genson/__init__.py: __version__ = '1.4.0rc1'
    
    # 2. Commit and push
    git commit -am "Bump version to 1.4.0rc1" && git push
    
    # 3. Tag and push
    git tag v1.4.0rc1 && git push origin v1.4.0rc1
  11. Manually verify a release (fallback)

    master

    If you need to manually verify a release (e.g., for a TestPyPI rc or a final PyPI release) outside of the automated GitHub Action jobs, use the following script. This creates a temporary virtual environment, installs the specific version from the target index, and runs basic checks.

    # Extract version from tag (assumes tag format v1.4.0)
    VERSION="$(git describe --tags --exact-match | sed 's/^v//')"
    VENV_DIR="$(mktemp -d)"
    
    # Setup venv
    python3 -m venv "$VENV_DIR/venv"
    
    # Install from TestPyPI (use --index-url for TestPyPI, or omit for PyPI)
    "$VENV_DIR/venv/bin/pip" install \
      --index-url https://test.pypi.org/simple/ "genson==$VERSION"
    
    # Verify import and version
    "$VENV_DIR/venv/bin/python" -c "import genson; print(genson.__version__)"
    
    # Verify CLI
    "$VENV_DIR/venv/bin/python" -m genson --version
    
    # Verify CLI functionality
    echo '{"hi": 5}' | "$VENV_DIR/venv/bin/genson"
    
    # Cleanup
    rm -rf "$VENV_DIR"