DOCSEARCHbyAlgolia
Submit Docs

python-benedict

repository·main·Indexed 23 days ago

https://github.com/fabiocaccamo/python-benedict

A dict subclass providing enhanced data access patterns including keypath, keylist, and keyattr support. It features normalized I/O operations for formats such as JSON, YAML, XML, TOML, CSV, and S3, along with utility methods for cleaning, flattening, and merging dictionaries. Optional Pydantic v2 integration allows for schema validation and type coercion during I/O operations.

Tokens
3K
Snippets
7
Records
13
Agent score
32%

What's inside python-benedict

  1. Overview of python-benedict features

    main

    python-benedict is a dict subclass designed for easier data manipulation. Key features include:

    • Backward Compatibility: It is a subclass of dict, so you can wrap existing dictionaries safely.
    • Keyattr: Access or set items using keys as attributes (e.g., my_dict.x.y).
    • Keylist: Access items using a list of keys (e.g., my_dict[['x', 'y']]).
    • Keypath: Access nested items using dot syntax (e.g., my_dict['x.y.z']).
    • List Index Support: Supports standard [n] syntax (including negative indices) within keypaths.
    • Normalized I/O: Built-in support for various formats including base64, csv, html, ini, json, pickle, plist, query-string, toml, xls, xml, and yaml.
    • Flexible Backends: I/O operations can target the file-system, url (read-only), or s3 (read/write).
    • Schema Validation: Optional Pydantic v2 validation for I/O methods via the schema keyword argument (requires python-benedict[schema]).

  2. Use keypaths and custom separators

    main

    By default, benedict uses . as a keypath separator to access nested items.

    • Custom Separator: You can change the separator using the keypath_separator argument in the constructor or via the property d.keypath_separator = "/".
    • Disabling: Set keypath_separator=None to disable keypath functionality.
    • List Indexing: Keypaths support list indexes (including negative ones) using [n] notation (e.g., "results[0].locations[-1]").

    Note: If you cast an existing dict that contains the separator in its keys, a ValueError or Exception will be raised.

    d = benedict()

# set values by keypath
d["profile.firstname"] = "Fabio"

# check if keypath exists
print("profile.firstname" in d)

# delete value by keypath
del d["profile.firstname"]

# List index support
loc = d["results[0].locations[-1].coordinates"]

  3. Use list of keys (Keylist)

    main

    Instead of a single key, you can pass a list of keys to perform operations on a specific nested path.

    d = benedict()

# set values by keys list
d[["profile", "firstname"]] = "Fabio"
d[["profile", "lastname"]] = "Caccamo"

# check if keypath exists in dict
print([["profile", "lastname"]] in d)

# delete value by keys list
del d[["profile", "lastname"]]

  4. Use attribute-style access (Keyattr)

    main

    You can get or set items using keys as attributes via dotted notation.

    • Dynamic access: If keyattr_dynamic=True is passed to the constructor, you can create new nested keys using attribute assignment.
    • Static access: By default (keyattr_dynamic=False), attribute access only works for keys that already exist.
    • Disabling: You can disable this feature using keyattr_enabled=False in the constructor or via the property d.keyattr_enabled = False.

    Warning: This only works for unprotected string keys (those not starting with _) that do not clash with existing method names.

    d = benedict(keyattr_dynamic=True)
d.profile.firstname = "Fabio"
d.profile.lastname = "Caccamo"
print(d) # -> { "profile":{ "firstname":"Fabio", "lastname":"Caccamo" } }

  5. Perform I/O operations with various formats

    main

    Benedict supports multiple input and output formats.

    Input

    Use class methods prefixed with from_* (e.g., from_json, from_yaml, from_xml). The first argument can be a file path, URL, S3 URL, or data string. You can restrict allowed sources using the sources argument (e.g., sources=['url']).

    Output

    Use instance methods prefixed with to_* (e.g., to_json, to_yaml). If the filepath keyword argument is provided, the output will be saved to that path.

    Schema Validation

    If you install the python-benedict[schema] extra, you can pass a Pydantic v2 model class to the schema= argument in from_* and to_* methods to validate and coerce data.

    # Input from S3
d = benedict("s3://my-bucket/data.xml", s3_options={"aws_access_key_id": "...", "aws_secret_access_key": "..."})

# Input with source restriction
d = benedict.from_json("https://localhost:8000/data.json", sources=["url"])

# Output to file
d.to_json(filepath="/path/to/file.json")

# Pydantic validation (requires python-benedict[schema])
from pydantic import BaseModel
class User(BaseModel):
    name: str
    age: int

d = benedict.from_json('{"name": "Alice", "age": "30"}', schema=User)
assert d["age"] == 30

  6. Install python-benedict

    main

    You can install the core package or a version containing all optional dependencies for full feature support (including various I/O formats and S3 support).

    To install the full suite of features, use the [all] extra. Otherwise, install the base package and add specific extras as needed.

  7. Initialize a benedict instance

    main

    Since benedict is a dict subclass, you can use it as a normal dictionary. You can create a new empty instance, cast an existing dictionary, or initialize it directly from a data source (filepath, URL, S3, or data string) in supported formats like JSON, YAML, TOML, XML, CSV, etc.

    from benedict import benedict

# create a new empty instance
d = benedict()

# or cast an existing dict
d = benedict(existing_dict)

# or create from data source (filepath, url or data-string)
d = benedict("https://localhost:8000/data.json", format="json")

# or in a Django view
params = benedict(request.GET.items())

  8. Run tests for python-benedict

    main

    To run the test suite for development purposes, you can use either tox or the standard unittest module. Before running tests, ensure you have installed the necessary requirements including requirements-test.txt and set up pre-commit hooks.

    # clone repository
git clone https://github.com/fabiocaccamo/python-benedict.git && cd python-benedict

# create virtualenv and activate it
python -m venv venv && . venv/bin/activate

# upgrade pip
python -m pip install --upgrade pip

# install requirements
pip install -r requirements.txt -r requirements-test.txt

# install pre-commit to run formatters and linters
pre-commit install --install-hooks

# run tests using tox
tox

# or run tests using unittest
python -m unittest

  10. Use Utility methods

    main

    Benedict provides several utility methods for common dictionary manipulations. Most methods that accept keys also support keypaths. Methods returning a dictionary always return a new benedict instance.

    • clean(strings=True, collections=True): Removes empty values (None, "", {}, [], etc.).
    • clone(): Returns a deepcopy of the dict.
    • dump(): Returns a readable representation of the dict/list.
    • filter(predicate, deep=True): Returns a filtered dict using a predicate function.
    • find(keys, default=0): Returns the first match for the given keys/keypaths.
    • flatten(separator="_", indexes=True): Returns a new flattened dict.
    • freeze(): Makes the dict immutable (top-level only).
    • merge(a, b, ..., overwrite=True, concat=False): Deep updates the current instance with other dicts.
    • remove(keys, deep=True): Removes multiple keys.
    • rename(old_key, new_key, deep=True): Renames a key.
    • standardize(): Standardizes all dict keys (e.g., to snake_case).

  11. Configure optional requirements for python-benedict

    main

    The package uses extras to manage optional dependencies. Installing a higher-level extra will automatically install all its sub-targets.

    Available installation targets:

    • [all]: Installs everything.
    • [io]: Installs I/O support, including:
      • [html]
      • [toml]
      • [xls]
      • [xml]
      • [yaml]
    • [parse]: Installs parsing utilities.
    • [s3]: Installs S3 read/write support.
    • [schema]: Installs Pydantic v2 schema validation and type coercion support for from_* and to_* methods.

  12. Reference: Supported I/O formats

    main

    The following table lists supported formats and their capabilities.

    | **format** | **input** | **output** |
|----------------|--------------------|--------------------|
| `base64` | :white_check_mark: | :white_check_mark: |
| `cli` | :white_check_mark: | :x: |
| `csv` | :white_check_mark: | :white_check_mark: |
| `html` | :white_check_mark: | :x: |
| `ini` | :white_check_mark: | :white_check_mark: |
| `json` | :white_check_mark: | :white_check_mark: |
| `pickle` | :white_check_mark: | :white_check_mark: |
| `plist` | :white_check_mark: | :white_check_mark: |
| `query-string` | :white_check_mark: | :white_check_mark: |
| `toml` | :white_check_mark: | :white_check_mark: |
| `xls` | :white_check_mark: | :x: |
| `xml` | :white_check_mark: | :white_check_mark: |
| `yaml` | :white_check_mark: | :white_check_mark: |