DOCSEARCHbyAlgolia
Submit Docs

StrictYAML

repository·master·Indexed 23 days ago

https://github.com/crdoconnor/strictyaml

A type-safe YAML parser for Python that validates a restricted subset of the YAML specification. It focuses on security and readability by enforcing strict type validation via schemas (using Map, Seq, Str, Int, etc.) and avoiding common pitfalls like the 'Norway problem'. StrictYAML prohibits explicit tags, node anchors, aliases, and flow style to ensure predictability and prevent vulnerabilities. It supports roundtripping to preserve comments and provides detailed YAMLError exceptions with line numbers.

Tokens
30.4K
Snippets
38
Records
206
Agent score
81%

What's inside strictyaml

  1. Explore StrictYAML usage guides and validators

    master

    StrictYAML provides a variety of guides for common YAML tasks, including building documents, schema validation, merging, and roundtripping. It also offers a rich set of compound and scalar validators to enforce strict data structures.

    Common Tasks

    • Building and Roundtripping: Build YAML documents from scratch, or read, edit, and write them back out (roundtripping).
    • Validation: Perform either/or schema validation, revalidate existing documents, and label exceptions.
    • Manipulation: Merge YAML documents and get line numbers for specific YAML elements.
    • Parsing: Parse YAML without a schema.

    Validator Types

    • Compound Validators: Used for complex structures like FixedSeq (fixed length sequences), Map (mappings with defined/optional/pattern keys), Seq (sequences), and UniqueSeq (sequences of unique items).
    • Scalar Validators: Used for individual values like Bool (booleans), Datetime, Decimal, Float, Int, HexInt, Enum (enumerated values), Regex (string regex validation), and Str (strings).

  2. Restrictions on combining Map or Seq validators with Or (|)

    master

    When using the | (Or) operator, you cannot combine two Map validators or two Seq validators directly. Attempting to do so will raise a strictyaml.exceptions.InvalidValidatorError. If you need to validate different structures of the same type, you should use revalidation instead.

    Invalid Map combination:

    # This raises InvalidValidatorError
load(yaml_snippet, Map({"a": Str()}) | Map({"b": Str()}))

    Invalid Seq combination:

    # This raises InvalidValidatorError
load(yaml_snippet, Seq(Int()) | Seq(Str()))
    load(yaml_snippet, Map({"a": Str()}) | Map({"b": Str()}))

  3. Why StrictYAML does not support node anchors and references

    master

    StrictYAML intentionally excludes support for YAML node anchors (&id) and references (*id). While these features allow for data deduplication within a YAML file, they often make the markup unreadable to non-programmers and can obscure the actual data structure.

    Instead of using anchors and references, StrictYAML encourages users to:

    1. Accept repetition: Use explicit, repeated data structures to maintain clarity.
    2. Refactor the schema: Redesign the data model to separate definitions from usage. For example, instead of using anchors to reference a block, define a definitions section and have your application logic resolve references (e.g., using a from key) during processing.

  4. Understand StrictYAML's approach to implicit typing

    master

    StrictYAML is designed to be a "zero surprises" parser by avoiding the implicit type conversion issues found in the YAML 1.2 specification. In many standard YAML parsers (like pyyaml or ruamel.yaml), certain strings are automatically converted into booleans, floats, or null values, which can lead to runtime errors in your application.

    StrictYAML avoids this by treating values as strings by default unless they are explicitly typed. This prevents common issues such as:

    • Boolean conversion: The string NO (Norway) being parsed as False.
    • Float conversion: Version numbers like 9.3 being parsed as a float instead of a string "9.3".
    • Null conversion: The string Null being parsed as the Python None type.

    By ignoring these parts of the spec, StrictYAML ensures that data remains in the format you expect without requiring manual quoting for every value.

  5. How parsed YAML mappings behave as YAML objects

    master

    When you parse a YAML document containing mappings using StrictYAML, the result is not a standard Python dict, but a specialized YAML object. This object provides dictionary-like behavior while maintaining StrictYAML's type-safe properties.

    Key capabilities of a parsed mapping object include:

    • Dictionary-like access: Use square bracket notation obj[key], .get(key, default), .items(), .keys(), and .values().
    • Membership testing: Use the in operator (e.g., "key" in obj) to check for key existence.
    • Length: Use len(obj) to get the number of keys.
    • Type checking: Use .is_mapping() to verify the object is a mapping.

    To convert the StrictYAML object into a plain Python dictionary (containing only standard dicts, lists, and primitive types), access the .data attribute.

  6. Schema enforcement during document updates

    master

    When a StrictYAML document is loaded using a schema, any subsequent updates to that document (via item assignment) are checked against the original schema. This ensures that modifications maintain the structural and type integrity defined during the initial load.

    Key behaviors:

    • Type Coercion: If you assign a value of a different type (e.g., a string '9' to an integer field), StrictYAML will attempt to coerce it to the schema-defined type.
    • Complex Types: Updates to complex types (like a field that can be s.Int() | s.Float()) will correctly resolve to the appropriate underlying data type.
    • Map Updates: You can update keys in a s.Map or s.MapPattern using standard dictionary-like assignment.
    import strictyaml as s

# Loading with a schema
doc = s.load('a: 9', s.Map({
  'a': s.Str(),
  s.Optional('b'): s.Int(),
}))

# Updating a value (coercing string '9' to int 9)
doc['b'] = '9'
assert doc['b'] == 9

  7. Understand StrictYAML's approach to syntax typing

    master

    StrictYAML avoids 'syntax typing'—the practice of using specific syntax (like quotation marks) to designate data types—in favor of schema-driven typing.

    In formats like JSON, types are embedded in the syntax (e.g., "11" is a string, 42 is an integer). In contrast, StrictYAML assumes all values are strings unless a schema explicitly defines them otherwise (e.g., using Map(Int(), Int())).

    Key behaviors in StrictYAML:

    • Implicit Conversion: You do not need quotation marks for strings that are intended to be converted to other types (e.g., yes or 1.5).
    • Required Quotation Marks: You only must use quotation marks for strings that are syntactically confusing to the parser, such as { text in curly brackets }.
    • Schema Dependency: Because the markup is terse, the responsibility for determining if a value is an integer, boolean, or string lies with the schema defined in your application code.

  8. Compare TOML and StrictYAML for configuration scaling

    master
    StrictYAML is designed for complex hierarchies and large-scale usage (e.g., 'story' tests with many files), whereas TOML is better suited for small, simple, and infrequent configuration tasks. As configuration scales, TOML's verbosity and syntax noise become more pronounced compared to StrictYAML's approach.

  9. Understand complexity and date/time handling

    master

    TOML includes built-in date and time parsing, which introduces significant complexity and edge cases into the format.

    StrictYAML handles complexity by decoupling it: the format itself is simple, and the library delegates specialized tasks like date/time validation to external tools (e.g., using Python's dateutil library). By default, StrictYAML parses values as strings, leaving validation to the schema layer.

  10. Understand the StrictYAML architecture

    master

    StrictYAML is not a new YAML standard, but a tool composed of two distinct parts designed to provide more predictable configuration parsing:

    1. A restricted YAML specification: It parses a subset of the YAML 1.2 specification. It is limited to parsing only into ordered dictionaries, lists, or strings.
    2. An optional validator: This component validates and casts scalar string values into specific types such as ints, floats, datetimes, etc.

    Because it follows YAML syntax, existing syntax highlighters and editors will recognize StrictYAML files as standard YAML. However, because StrictYAML is stricter, not all standard YAML files will be compatible with it.

  11. Use MapPattern for mappings with arbitrary key names

    master

    Use MapPattern when you want to validate the types of keys and values in a mapping without requiring specific, fixed key names. This is useful for dictionaries where the keys are dynamic but must follow a certain type (e.g., all keys are strings and all values are integers).

    If you need to enforce exact key names, use the Map validator instead of MapPattern.

  12. Compare Kwalify vs StrictYAML for schema validation

    master

    When deciding between using Kwalify with standard YAML or using StrictYAML, consider the complexity of your validation requirements. Kwalify is a descriptive schema language written in YAML that is suitable for simple YAML validation, but it has significant limitations compared to StrictYAML.

    Limitations of Kwalify compared to StrictYAML:

    • No external data injection: You cannot plug in generated lists from external sources (e.g., a list of country codes from a library like pycountry) into your spec.
    • Limited polymorphism: You cannot validate parts of the schema that can be multiple types (e.g., a field that can be either a single string OR a list of strings).
    • No sub-validator composition: You cannot plug sub-validators of a document into larger, more complex validators.

    When to use Kwalify:

    • Your schema is very simple and small.
    • Your schema validation requirements must be shared with a third party, especially one using a different programming language.

    When to use StrictYAML:

    • Your schema validation requirements are complex or require the advanced features listed in the limitations above.