xmlschema

repository·master·Indexed 19 days ago

https://github.com/sissaschool/xmlschema

A Python library for implementing XML Schema (XSD) 1.0 and 1.1. It provides tools for validating XML instances, encoding and decoding XML to and from Python data and JSON, and navigating schemas via XPath. The library includes support for various converters (such as BadgerFish and JsonML), resource management for remote schemas, and detailed inspection of XSD components like elements, attributes, and facets.

Tokens
26.2K
Snippets
86
Records
127
Agent score
67%

What's inside xmlschema

  1. Choose an XML converter convention

    master

    The xmlschema library uses converter classes to handle the decoding (XML to intermediate data) and encoding (intermediate data to XML) of XML data. Because XML contains metadata like attributes and namespaces, converters apply specific naming conventions to represent these in formats like JSON.

    Predefined Converters

    Depending on your target data format or preferred convention, you can use one of the following derived classes:

    • xmlschema.ParkerConverter: Implements the Parker convention.
    • xmlschema.BadgerFishConverter: Implements the BadgerFish convention.
    • xmlschema.AbderaConverter: Implements the Apache Abdera project convention.
    • xmlschema.JsonMLConverter: Implements the JsonML (JSON Mark-up Language) convention.
    • xmlschema.UnorderedConverter: Similar to the default converter but performs unordered decoding and encoding.
    • xmlschema.ColumnarConverter: Remaps attributes as child elements in a columnar shape.
    • xmlschema.DataElementConverter: Converts XML to a tree of xmlschema.DataElement instances (Element-like objects with decoded values and schema bindings).

    Base Class Options

    All converters inherit from xmlschema.XMLSchemaConverter. While specific converters may fix certain values, the base class supports options to vary the conversion process, such as force_list and force_dict (though these may be ignored by some predefined converters).

  2. Manage global schema settings with SchemaSettings

    master

    Starting from v4.2, schema instances are configured using xmlschema.settings.SchemaSettings instances. These settings control how schema and XML resources are handled securely.

    When creating a new schema, the instance inherits settings from the global defaults, which can be overridden by providing arguments to xmlschema.XMLResource (passed as optional keyword arguments during schema creation).

    To ensure multiple schema instances share the same configuration, use the XMLSchemaBase.from_settings method.

    To manage configuration globally for an entire application, you can modify the library-wide defaults using update_defaults or revert them using reset_defaults.

    import xmlschema
    from xmlschema.settings import SchemaSettings
    
    # Update global defaults for all subsequent schema creations
    SchemaSettings.update_defaults(some_setting=True)
    
    # Restore to library defaults
    SchemaSettings.reset_defaults()
    
    # Create a schema using specific settings
    schema = xmlschema.XMLSchema(schema_path, some_setting=True)
    
    # Create a new schema instance based on an existing one's settings
    new_schema = schema.from_settings(another_schema_path)
  3. Configure XSD validation modes (strict, lax, skip)

    master

    The validation argument controls how the processor handles schemas and XML data. The default mode is strict.

    • strict: Validates schemas against the meta-schema. Stops execution immediately upon finding any error in the schema or XML data.
    • lax: Validates schemas against the meta-schema. Collects errors and continues; missing parts are replaced with wildcards, and undecodable XML data is replaced with None.
    • skip: Does not validate schemas against the meta-schema and does not collect errors. Undecodable XML data is replaced with its original text.

    Note: For iter_decode() and iter_encode(), errors are propagated even in skip mode, though top-level decode() and encode() methods discard them.

    import xmlschema
    
    schema = xmlschema.XMLSchema('schema.xsd', validation='strict')
    # You can override the mode during decoding
    data = schema.decode(xml_data, validation='lax')
  4. Customize the decoded data structure with Converters

    master

    You can control how XML elements are transformed into Python structures by providing a converter instance. This is useful for switching between different data formats like BadgerFish or Parker.

    Converters can be specified:

    1. During XMLSchema instantiation via the converter argument.
    2. During a specific method call (like to_dict) via the converter argument.
    import xmlschema
    
    # Use BadgerFishConverter for the whole schema instance
    xs = xmlschema.XMLSchema('schema.xsd', converter=xmlschema.BadgerFishConverter)
    
    # Or use ParkerConverter for a single decoding operation
    data = xs.to_dict('data.xml', converter=xmlschema.ParkerConverter)
  5. Identify XML content types

    master

    An element can fall into one of four content categories. You can check these using the following methods:

    • empty: Denies child elements and text content. Use is_empty().
    • simple: Denies child elements but allows text content. Use has_simple_content().
    • element-only: Allows child elements but denies intermingled text. Use is_element_only().
    • mixed: Allows both child elements and intermingled text. Use has_mixed_content().

    To simplify determining the validator, XSD types provide two helper properties:

    • simple_type: Returns the simple type if the content is simple or empty (based on a simple type), otherwise None.
    • model_group: Returns the model group if the content is mixed or element-only (or empty based on a model group), otherwise None.
  6. Inspect XSD Simple and Complex types

    master

    Every element or attribute has a .type attribute.

    Simple Types

    Used for attributes and elements containing only text.

    • Use .is_simple() to check if a type is simple.
    • Simple types do not have an .attributes property.
    • They may have .validators (e.g., for facets like positiveInteger) and properties like .white_space.

    Complex Types

    Used for elements with attributes or child elements.

    • Use .has_complex_content() to check if it is a complex type.
    • Access attributes via the .attributes property (which returns an XsdAttributeGroup).
    • Access the content model via the .content property (usually an XsdGroup).
    • Use .iter_elements() on the .content property to traverse nested elements within a model group.
    # Inspecting a complex type
    person = schema.elements['person']
    print(person.type.has_complex_content())
    print(person.type.content)  # Returns XsdGroup
    
    # Traversing nested elements in a complex type
    for e in person.type.content.iter_elements():
        print(e)
    
    # Inspecting a simple type
    step_attr = schema.attributes['step']
    print(step_attr.type.is_simple())
    print(step_attr.type.validators)
  7. Generate source code from XSD using AbstractGenerator

    master

    The xmlschema.extras.codegen module provides the AbstractGenerator base class to generate source code from parsed XSD schemas using the Jinja2 engine. When implementing a custom generator, you can leverage built-in schema-based filters and tests within your Jinja2 templates to access XSD components.

    from xmlschema.extras.codegen import AbstractGenerator
    
    # Implementation would involve subclassing AbstractGenerator and providing templates
    class MyGenerator(AbstractGenerator):
        formal_language = 'MyLanguage'
        # ... implementation details ...
  8. Use XSD 1.1 validators for advanced schema features

    master
    For schemas utilizing XSD 1.1 features, use the classes in the xmlschema.validators module. These include specialized classes for elements (Xsd11Element), attributes (Xsd11Attribute), complex types (Xsd11ComplexType), unions (Xsd11Union), and identity constraints (Xsd11Unique, Xsd11Key, Xsd11Keyref).
  9. Protect against XML entity-based attacks with 'defuse'

    master

    The defuse argument in XMLSchema regulates protection against XML entity-based attacks (using SafeXMLParser).

    • 'remote' (Default): Protection is applied only to XML data loaded from remote sources.
    • 'nonlocal': All XML data is defused except for local files.
    • 'always': All XML data is defused.
    • 'never': No protection is applied.

    Security Recommendation: For public-facing services (like online validators), use defuse='always' and allow='none' to prevent filesystem attacks via direct paths or XSD injection.

  10. Handle Decimal serialization when decoding to JSON

    master

    When using xmlschema.to_dict() to convert XML data to a dictionary for JSON serialization, XSD decimal types are converted to Python Decimal objects. Since Decimal is not JSON serializable by default, you should use the decimal_type keyword argument to specify a JSON-compatible type, such as str.

    import xmlschema
    import json
    
    # This will fail if the XML contains decimal values
    # json.dumps(xmlschema.to_dict(xml_document))
    
    # Use decimal_type=str to ensure compatibility
    json_data = json.dumps(xmlschema.to_dict(xml_document, decimal_type=str), indent=4)
    print(json_data)
    import xmlschema
    import json
    
    xml_document = 'tests/test_cases/examples/collection/collection.xml'
    # Use decimal_type=str to avoid TypeError: Decimal(...) is not JSON serializable
    print(json.dumps(xmlschema.to_dict(xml_document, decimal_type=str), indent=4))
  11. Extend tests with custom XSD and XML files

    master

    You can dynamically create test cases from your own XSD and XML files by creating a testfiles index file.

    1. Create a directory (outside the repository or as a submodule) containing your XSD/XML files.
    2. Create an index file named testfiles in that directory.
    3. List the paths to your files in the testfiles index, one per line. You can include comments and specify expected error counts.

    Example testfiles content:

    # XHTML
    XHTML/xhtml11-mod.xsd
    
    # Quantum Espresso
    qe/qes.xsd
    qe/silicon.xml
    qe/silicon-1_error.xml --errors 1

    To run the tests using your custom index, provide the path to the testfiles index to the test script:

    python xmlschema/tests/test_all.py ../extra-schemas/testfiles