remarshal

While Remarshal is primarily a CLI application, it can be used as a Python library. Because the library usage is not as strictly governed as the CLI, it is recommended to pin your dependency to a specific minor version to avoid breaking changes in the Python API during minor updates.

Example dependency constraint: remarshal>=2.1,<2.2

Note that dropping support for old Python versions is not considered a breaking change and will not trigger a major version bump.

# In your requirements.txt or pyproject.toml
remarshal>=2.1,<2.2

YAML tags (e.g., !secret) are not natively supported by other formats like JSON. By default, Remarshal errors when encountering tags.

Using --yaml-tags allows you to represent a tag as a single-key mapping where the key starts with !. This enables round-tripping tags through formats that don't support them.

Example: Converting tagged YAML to JSON

$ printf 'name: !secret pw\nport: 8123\n' | remarshal --from yaml --to json --yaml-tags
{"name":{"!secret":"pw"},"port":8123}

$ echo '{"!my-tag": {"foo": "bar", "baz": 123}}' \
  | remarshal --from json --to yaml --yaml-tags
!my-tag
foo: bar
baz: 123

TOML requires the top-level element to be a dictionary (map). If your input data (from JSON, CBOR, etc.) is a list or a primitive, conversion to TOML will fail.

• --wrap <key>: Wraps the input data in a new dictionary where the input is the value associated with <key>.
• --unwrap <key>: Extracts the value associated with <key> from the top-level dictionary and outputs only that value, discarding the rest.

Example: Converting a JSON list to TOML

# This fails:
echo '[{"a":"b"}]' | remarshal --from json --to toml

# This works by wrapping the list in a 'main' key:
echo '[{"a":"b"}]' | remarshal --from json --to toml --wrap main

$ echo '[{"a":"b"},{"c":[1,2,3]}]' \
  | remarshal --from json --to toml --wrap main
[[main]]
a = "b"

[[main]]
c = [1, 2, 3]

By default, Remarshal operates in lossless mode. It attempts to ensure that a document converted from format A to B and back to A remains identical to the original. If a lossless conversion is impossible, Remarshal will exit with an error.

To relax this restriction and allow conversions that might lose type fidelity (by converting specific types to strings), use the -k or --stringify option. This is often necessary when converting to JSON or TOML.

When --stringify is enabled:

• JSON conversion: Boolean, null, and date-time keys/values are turned into strings.
• TOML conversion: Boolean, date-time, and null keys/values are turned into strings.
• Date-time to JSON: Converting documents with date-time types to JSON requires this option to turn them into strings.

remarshal -k [args]

Remarshal requires Python 3.12 or later. The recommended way to install it is using pipx or uv to ensure it is available as a standalone tool.

To install the latest release:

pipx install remarshal
# or
uv tool install remarshal

You can also run Remarshal without a permanent installation using pipx run or uvx, which will download and cache the tool temporarily:

pipx run remarshal [arg ...]
# or
uvx remarshal [arg ...]

To install the current development version from GitHub:

pipx install git+https://github.com/remarshal-project/remarshal
# or
uv tool install git+https://github.com/remarshal-project/remarshal

Remarshal is a CLI tool for converting between CBOR, JSON, MessagePack, TOML, and YAML. You can use the main remarshal command with explicit format flags or use short-hand commands like json2yaml or yaml2toml.

Input/Output:

• If no input file is provided or -i - is used, Remarshal reads from stdin.
• If no output file is provided or -o - is used, Remarshal writes to stdout.

Exit Codes:

• 0: Success
• 1: Operational failure
• 2: Command line parsing failure

# Using explicit flags
remarshal -f json -t yaml input.json output.yaml

# Using short-hand commands
json2yaml input.json output.yaml

# Using stdin/stdout
echo '{"a": 1}' | json2yaml - > output.yaml

Remarshal can transform data into Python code. This is a one-way operation.

Output Formats

• Default: Uses Python's repr() format. Note that the default repr format ignores the -s/--sort-keys flag.
• Formatted: Uses pprint.pformat when the --indent option is passed.

Important Notes

• Imports: The generated Python code does not include necessary import statements. For example, if your data contains date-times, you must manually add import datetime to your script.
• Formatting: The pprint output style may not match your project's specific coding style; it is recommended to run a dedicated Python formatter (like Black or Ruff) on the output.

remarshal --indent [input_file] > output.py

In Remarshal, YAML tags (like !secret) are represented in-memory using the TaggedValue dataclass. This allows you to preserve tag information during transformations.

When converting to formats that do not support native tags (like JSON), TaggedValue is typically converted into a single-key mapping envelope, e.g., {"!secret": "password"}. This behavior is controlled by the --yaml-tags option in the CLI.

To convert a document containing TaggedValue objects into these mapping envelopes (useful before encoding to non-YAML formats), use tags_to_envelopes(). To convert these envelopes back into TaggedValue objects (useful before encoding to YAML), use envelopes_to_tags().

from remarshal.document import TaggedValue

tagged = TaggedValue(tag="!secret", value="password")
# Represented as: TaggedValue(tag='!secret', value='password')

Importing the remarshal.codecs package automatically registers all built-in codecs into the remarshal.codec registries (DECODERS and ENCODERS). This allows the core remarshal library to automatically discover and use these codecs when performing conversions between formats.

import remarshal.codecs
# All codecs are now registered and available via the remarshal registry.

Remarshal supports both YAML 1.1 and YAML 1.2. By default, it uses YAML 1.2. You can explicitly select the version using the following format identifiers:

• yaml-1.1: Use YAML 1.1.
• yaml-1.2: Use YAML 1.2 (default).

Conversion between these two versions is supported.