cattrs

repository·main·Indexed 21 days ago

https://github.com/python-attrs/cattrs

A library for structuring and unstructuring data in Python, specializing in converting unstructured dictionaries into structured classes (such as attrs or dataclasses) and back again. It supports nested data, complex types, and custom converters via hooks and hook factories. The package includes cattrs.preconf for optimized integration with serialization formats like JSON, YAML, Msgpack, BSON, and TOML.

Tokens
29.5K
Snippets
87
Records
107
Agent score
73%

What's inside cattrs

  1. Use cattrs.preconf for specialized serialization formats

    main

    The cattrs.preconf package provides pre-configured converters for various serialization formats. Instead of manually configuring a cattrs.Converter to work with specific libraries, you can use these submodules to get a converter that is already optimized and configured for the target format (e.g., JSON, YAML, TOML, Msgpack, etc.).

    Available submodules include:

    • cattrs.preconf.bson (BSON)
    • cattrs.preconf.cbor2 (CBOR)
    • cattrs.preconf.json (Standard JSON)
    • cattrs.preconf.msgpack (MessagePack)
    • cattrs.preconf.msgspec (msgspec)
    • cattrs.preconf.orjson (orjson)
    • cattrs.preconf.pyyaml (PyYAML)
    • cattrs.preconf.tomlkit (TOMLKit)
    • cattrs.preconf.tomllib (Standard TOML)
    • cattrs.preconf.ujson (ujson)
  2. What is the difference between structured and unstructured data in cattrs?

    main

    In the context of cattrs, data is categorized into two types:

    1. Unstructured data: Low-level, built-in Python types like dict, list, tuple, int, str, etc. These are easy to serialize to formats like JSON, YAML, or MessagePack but lack the validation and constraints required for business logic.
    2. Structured data: Well-defined classes (such as those created with attrs or dataclasses) that represent your business logic. These provide type safety and ensure that data conforms to expected shapes.

    cattrs acts as the bridge between these two worlds: it converts unstructured data into trustworthy structured data (structuring) and converts structured classes back into primitive types for serialization (unstructuring).

  3. Use `cattrs.gen` hook factories for high-performance customization

    main

    The cattrs.gen module provides hook factories for generating specialized un/structuring hooks for attrs classes, dataclasses, and TypedDicts.

    Using these factories instead of standard cattrs machinery offers two main benefits:

    1. Performance: Generated hooks can bypass much of the standard cattrs machinery, making them significantly faster.
    2. Granular Control: They allow for overriding behavior on a per-attribute basis (e.g., renaming a field or omitting it).

    Commonly used factories include make_dict_unstructure_fn and make_dict_structure_fn. Once a hook is generated, you must register it with a cattrs.Converter using register_unstructure_hook or register_structure_hook.

    from cattrs.gen import make_dict_unstructure_fn, override
    
    # Generate a specialized hook
    hook = make_dict_unstructure_fn(MyClass, converter, field_name=override(rename="new_name"))
    
    # Register it
    converter.register_unstructure_hook(MyClass, hook)
  4. Configure the msgspec JSON converter

    main

    The cattrs.preconf.msgspec converter is optimized for msgspec. It is currently considered provisional.

    Key behaviors:

    • Strict Mode: Enabled by default. You can customize this by modifying the converter.encoder attribute.
    • Structs: msgspec structs are supported but not composable; a struct is handed over to msgspec directly for recursive handling.
    • Validation: Because msgspec handles many types directly, validation errors might be msgspec errors instead of cattrs errors.
    • Optimization: You can use get_dumps_hook(type) and get_loads_hook(type) to obtain highly optimized functions that offload as much work as possible to msgspec.
    from cattrs.preconf.msgspec import make_converter
    from attrs import define
    
    @define
    class Test:
        a: int
    
    converter = make_converter()
    # Get an optimized dump function for the Test type
    dumps = converter.get_dumps_hook(Test)
    
    # This will use msgspec directly for maximum performance
    binary_data = dumps(Test(1))
  5. How converters work in cattrs

    main

    Converters are registries of rules that cattrs uses to perform function composition and generate unstructuring (converting objects to primitives) and structuring (converting primitives to objects) functions.

    A converter maintains several pieces of state:

    • Unstructure hooks: A registry of rules for converting objects to primitives, using singledispatch and FunctionDispatch with caching.
    • Structure hooks: A registry of rules for converting primitives to objects, also using singledispatch, FunctionDispatch, and caching.
    • detailed_validation flag: A boolean (defaults to true) that determines if the converter uses detailed validation.
    • Unstructuring strategy: An UnstructureStrategy (either AS_DICT or AS_TUPLE).
    • prefer_attrib_converters flag: A boolean (defaults to false) that determines whether to favor attrs converters over normal cattrs machinery when structuring attrs classes.
    • dict_factory: A legacy parameter used for creating dicts when dumping attrs classes using AS_DICT.

    You can create a modified version of an existing converter using the Converter.copy() method. The new copy will retain all manually registered hooks from the original but can be modified via the copy arguments.

  6. Structure and unstructure Tuples

    main

    Tuples can be structured from iterable objects. There are two types:

    1. Heterogeneous Tuples: Use typing.Tuple[A, B, ...] or tuple[A, B, ...]. The input iterable must match the number of type parameters exactly.
    2. Homogeneous Tuples: Use collections.abc.Sequence[T], typing.Tuple[T, ...], or tuple[T, ...].

    Note:

    • As of version 25.2.0, abstract Sequence types are structured into tuple.
    • Unstructuring heterogeneous tuples results in a tuple for performance and compatibility.
    • BaseConverter does not support structuring heterogeneous tuples.
    >>> import cattrs
    >>> cattrs.structure([1, 2, 3], tuple[int, str, float])
    (1, '2', 3.0)
  7. How the default union structuring strategy works

    main

    When structuring a Union of attrs classes or dataclasses, cattrs uses an opinionated default strategy to disambiguate which class to instantiate. It follows these steps in order:

    1. Literal Fields: If all members of the union contain a typing.Literal field, cattrs uses that field to determine the correct class. For example, if ClassA has field: Literal['one'] and ClassB has field: Literal['two'], a payload {'field': 'one'} will be structured as ClassA.
    2. Unique Required Fields: If no suitable Literal fields exist, cattrs looks for fields that are required (i.e., have no default value) and unique to each class in the union. If a payload contains a key that is a required field in ClassA but not in ClassB, it will be structured as ClassA.

    Note: Fields with default values are treated as optional and are not used for disambiguation.

    from typing import Literal
    from attrs import define
    
    @define
    class ClassA:
        field_one: Literal["one"]
    
    @define
    class ClassB:
        field_one: Literal["two"] = "two"
    
    # A payload of {"field_one": "one"} will produce ClassA
  8. Understand detailed validation mode

    main

    Since version 22.1.0, cattrs uses a detailed validation mode by default. In this mode, structuring hooks are slightly slower but provide richer, more precise error messages by grouping errors into trees of exceptions.

    Key behaviors:

    • Grouping: Errors are gathered on a field-by-field, key-by-key, or index-by-index basis.
    • Exception Types: Errors are raised as cattrs.BaseValidationError (a PEP 654 ExceptionGroup).
      • cattrs.ClassValidationError: Raised when errors occur while structuring a class (subclass of BaseValidationError).
      • cattrs.IterableValidationError: Raised when errors occur while structuring sequences or mappings (subclass of BaseValidationError).
    • Metadata: Exceptions include __notes__ (per PEP 678) indicating the specific field, key, or index where the error occurred.
    • Unstructuring: Unstructuring hooks are not affected by detailed validation mode.
  9. How recursive structuring works

    main

    Structuring converts unstructured data (like a dict) into structured objects based on a provided type specification. cattrs supports a wide range of types recursively:

    • Optionals: typing.Optional[T] and T | None.
    • Collections: list[T], tuple, set[T], frozenset[T], and dict[K, V] (including various typing variants).
    • Special Types: typing.TypedDict, typing.NewType, and PEP 695 type aliases (Python 3.12+).
    • Classes:
      • attrs classes with simple attributes.
      • All attrs classes and dataclasses if complex attributes have type metadata.
    • Unions:
      • Unions of supported attrs classes (if they have unique fields).
      • Any union, provided you supply a disambiguation function.

    You can extend this by registering custom converters using register_structure_hook.

  10. Structure and unstructure TypedDicts

    main

    TypedDicts can be structured from mapping objects (usually dictionaries). Both total and non-total TypedDicts, as well as inheritance patterns, are supported. Generic TypedDicts require Python 3.11+.

    Customization: You can customize un/structuring using cattrs.gen.typeddicts.make_dict_structure_fn and cattrs.gen.typeddicts.make_dict_unstructure_fn. This is useful for renaming keys during structuring.

    Warning: If from __future__ import annotations is used or if annotations are provided as strings, typing.Required and typing.NotRequired are ignored by cattrs.

    from typing import TypedDict
    from cattrs import Converter
    from cattrs.gen import override
    from cattrs.gen.typeddicts import make_dict_structure_fn
    
    class MyTypedDict(TypedDict):
        a: int
        b: int
    
    c = Converter()
    c.register_structure_hook(
        MyTypedDict,
        make_dict_structure_fn(
            MyTypedDict,
            c,
            a=override(rename="a-with-dash")
        )
    )
    
    # Maps 'a-with-dash' in input to 'a' in the TypedDict
    print(c.structure({"a-with-dash": 1, "b": 2}, MyTypedDict))
    # {'b': 2, 'a': 1}
  11. Structure and unstructure Sets and Frozensets

    main

    Sets and frozensets can be structured from any iterable object.

    Supported Types:

    • Sets: set[T], typing.Set[T], or collections.abc.MutableSet[T].
    • Frozensets: frozenset[T], typing.FrozenSet[T], or collections.abc.Set[T].

    Note: As of version 25.3.0, abstract sets are structured into frozenset instead of set.

    When unstructuring, sets and frozensets are converted back to their matching class.

    >>> import cattrs
    >>> cattrs.structure([1, 2, 3, 4], set)
    {1, 2, 3, 4}