python-toon (Token-Oriented Object Notation)

By default, the decoder validates input strictly. Strict mode checks for:

• Invalid escape sequences: Throws on "\x", unterminated strings
• Syntax errors: Throws on missing colons, malformed headers
• Array length mismatches: Throws when declared length doesn't match actual count
• Delimiter mismatches: Throws when row delimiters don't match header

Set strict=False in DecodeOptions to allow lenient parsing.

TOON uses specific syntax for different data structures:

• Objects: Key-value pairs with colons (key: value).
• Primitive Arrays: Arrays include a length indicator [N]. For primitive arrays, the delimiter is hidden (e.g., [3]: val1,val2,val3).
• Tabular Arrays: Uniform objects with identical primitive-only fields use a CSV-like format. The delimiter is shown in the header (e.g., [2,]{id,name}:).
• Mixed Arrays: Non-uniform data uses a list format with - markers for each element.
• Quoting: Strings are quoted only when necessary (e.g., containing structural characters, leading/trailing whitespace, or keywords like null, true, false).

If you prefer not to use GitHub Actions, you can build and upload the package manually using build and twine.

1. Install build tools:

   pip install build twine

2. Build the package:

   python -m build

3. Upload to TestPyPI (for testing):

   twine upload --repository testpypi dist/*

   • Username: __token__
   • Password: Your TestPyPI API token

4. Upload to PyPI (for production):

   twine upload dist/*

   • Username: __token__
   • Password: Your PyPI API token

Use the encode function to convert Python dictionaries or lists into TOON format. TOON supports simple objects, tabular arrays (for uniform objects), and complex nested structures.

from toon import encode

# Simple object
data = {"name": "Alice", "age": 30}
print(encode(data))
# Output:
# name: Alice
# age: 30

# Tabular array (uniform objects)
users = [
    {"id": 1, "name": "Alice", "age": 30},
    {"id": 2, "name": "Bob", "age": 25},
    {"id": 3, "name": "Charlie", "age": 35},
]
print(encode(users))
# Output:
# [3,]{id,name,age}:
#   1,Alice,30
#   2,Bob,25
#   3,Charlie,35

To get the best results when using TOON with LLMs:

1. Wrap in code blocks: Use ```toon for clarity.
2. Provide explicit instructions: Tell the model: "Respond using TOON format (Token-Oriented Object Notation). Use key: value syntax, indentation for nesting, and tabular format [N,]{fields}: for uniform arrays."
3. Use length markers for validation: Enable lengthMarker: "#" during encoding and instruct the model: "Array lengths are marked with [#N]. Ensure your response matches these counts."

Install the python-toon package using pip to use the TOON encoder/decoder in your Python projects.

pip install python-toon

The recommended way to publish python-toon is using GitHub Actions with Trusted Publishing (OIDC). This method does not require storing secrets in GitHub.

To PyPI (Production)

1. Create a new release on GitHub.
2. Include a version tag (e.g., v0.1.0).
3. GitHub Actions will automatically build and publish the package to PyPI.

To TestPyPI (Testing)

1. Navigate to the Actions tab in the GitHub repository.
2. Select the Publish to PyPI workflow.
3. Click Run workflow.
4. Select your branch and click Run workflow to trigger the test upload.

When preparing a new release, you must update the version number in two specific locations to ensure consistency:

1. pyproject.toml (Line 7)
2. src/toon/__init__.py (Line 11)

Failure to update both files may result in version mismatches between the package metadata and the runtime __version__ attribute.

"Project not found" on PyPI

• Ensure Trusted Publishing is configured on PyPI.
• Verify the project name matches exactly: python-toon.
• Ensure the pending publisher was created before the first publish attempt.

"Environment protection rules" error

• Verify GitHub environment settings.
• Ensure you have the necessary permissions to deploy to the environment.
• If the environment is protected, an admin must approve the deployment.

"OIDC token verification failed"

• Confirm the workflow name is exactly publish.yml.
• Confirm the environment name matches exactly (pypi or testpypi).
• Verify the GitHub username/organization matches the Trusted Publishing configuration.

Converts a JSON-serializable Python value to a TOON-formatted string.

from toon import encode

data = {"id": 123, "name": "Ada"}
toon_str = encode(data)
print(toon_str)

Use the DecodeOptions class to control the decoding behavior, such as indentation and strictness.

from toon import decode, DecodeOptions

options = DecodeOptions(
    indent=2,    # Expected number of spaces per indentation level (default: 2)
    strict=True  # Enable strict validation (default: True)
)

data = decode(toon_str, options)