DOCSEARCHbyAlgolia
Submit Docs

Rapid YAML (ryml)

repository·master·Indexed 20 days ago

https://github.com/biojppm/rapidyaml

A high-performance C++11 library for parsing and emitting YAML and JSON. It is optimized for speed and minimal memory overhead using a flat, index-based data tree and view-based parsing to avoid string copies. The library is YAML 1.2 compliant, exception-agnostic, and supports a wide range of platforms from x64 to bare-metal systems. It is available for C++, Python, and JavaScript (via WebAssembly).

Tokens
40.4K
Snippets
122
Records
199
Agent score
69%

What's inside rapidyaml

  1. Overview of Rapid YAML (ryml)

    master

    ryml (or Rapid YAML) is a high-performance C++11 library designed for parsing and emitting YAML and JSON. It is optimized for speed and efficiency across a wide range of platforms, from x64 to bare-metal systems without an operating system.

    Key Features:

    • High Performance: Designed for massive datasets with extremely fast parsing and serialization.
    • Memory Efficient: Uses a flat, index-based data tree. The tree holds views to sub-ranges of the source buffer, avoiding string copies or duplications.
    • YAML 1.2 Compliant: Fully conformant to the YAML 1.2 specification.
    • Robustness: Uses a non-recursive, state-machine-based parser. It is designed to be robust and is extensively tested/fuzzed.
    • Flexible Memory & Error Handling: Supports custom global and per-tree memory allocators and error handler callbacks. It is exception-agnostic.
    • No STL Dependency: The core data structures do not depend on the C++ Standard Template Library (STL), though it can be used to serialize/deserialize STL containers.

  2. Compare rapidyaml performance with other libraries

    master

    rapidyaml is designed for high-performance YAML and JSON processing.

    YAML Performance

    Compared to yamlcpp, libyaml, and fyaml, rapidyaml is significantly faster, typically providing a 30x speedup for parsing and a 150x speedup for emitting. In absolute terms, it achieves approximately 200MB/s for parsing and 600MB/s for emitting.

    JSON Performance

    rapidyaml is also a top-tier JSON handler.

    • Parsing: ryml_json achieves ~908 MB/s. It competes closely with rapidjson_inplace (~1741 MB/s) and is significantly faster than nlohmann, jsoncpp, and sajson.
    • Emitting: ryml_json_str achieves ~1034 MB/s, making it one of the fastest emitters available, exceeding rapidjson, jsoncpp, and nlohmann.

  3. Understand NodeRef states: invalid, readable, and seed

    master

    In version 0.6.0, the state of a NodeRef was refined into three mutually exclusive states. You should use these predicates instead of the deprecated .valid() method to determine the status of a node reference:

    • .invalid(): The object was not initialized to any node.
    • .readable(): The object points to an existing tree and a valid node.
    • .is_seed(): The object points to a hypothetical tree/node (a 'seed').

    Note: .valid() is deprecated because its semantics were ambiguous, as it could represent either a .readable() or an .is_seed() state.

    // Use these instead of .valid()
bool readable() const { return valid() && !is_seed(); }

  4. Use id_type instead of size_t for node IDs

    master
    A new type id_type has been introduced for node identifiers. While it currently defaults to size_t for backward compatibility, it may change to a signed type in the future. To ensure future-proof code, always use id_type instead of size_t when handling node IDs.

  5. Use TagCache to optimize heavily-tagged YAML

    master
    For YAML files that use a high volume of tags, version 0.12.0 introduced TagCache (located in c4/yml/tag.hpp). This accelerator structure is used by both the ParseEngine and Tree::resolve_tags() to ensure reuse of resolved tags, significantly reducing arena memory requirements.

  6. Use id_type for node IDs

    master
    RapidYAML uses id_type to represent node IDs. While it currently defaults to size_t for backward compatibility, it is expected to change (potentially to a signed type) in future versions. To ensure future-proof code, always use id_type instead of size_t when handling node IDs.

  7. Configure error handling behavior in C++

    master

    Rapid YAML's error handling behavior changes based on your build configuration and macros:

    1. With Assertions Enabled: Errors (such as invalid node access) will trigger assertions. If an assertion calls an error-throwing callback, it will throw an exception.
    2. In Release Builds (Assertions Disabled): By default, assertions are disabled in release builds. Warning: User code that relies on invalid operations will silently succeed and return garbage data in release builds. To prevent this, use the at() methods for checked access.
    3. Exception Handling: To make the library throw exceptions instead of aborting the process when an error occurs, define the macro RYML_DEFAULT_CALLBACK_USES_EXCEPTIONS.

  9. Manage external c4core source files

    master

    The ext folder contains source files from the external c4core project. To avoid using git submodules, these files are explicitly copied into rapidyaml. Synchronization is managed via c4core.mk (which defines the version), a Makefile (providing sync targets), and sync.py (the underlying helper script).

    Key Files

    • c4core.mk: Specifies the c4core repository and the specific version to sync. The version must be a git tag (e.g., v0.4.0) or a full SHA1 hash (e.g., 2d123704c0594f818162324ef7805a8cc6f5895d). Short hashes or branch names are not supported.
    • Makefile: Provides targets for checking sync status and performing imports/exports.
    • sync.py: The core helper script used by the Makefile.
    • c4core.src: Folder containing c4core source files used by the rapidyaml library.
    • c4core.dev: Folder containing c4core files used only for internal development.
    • c4core.cmake: CMake functions used to inject these source files into CMake targets based on the manifests (c4core.src.txt, c4core.dev.txt, or c4core.yml).

  10. Handle null keys and values in v0.8.0+

    master

    In version 0.8.0, the way empty keys and values are handled changed. New NodeTypeBits were added to mark them, and new methods were introduced to check for nullity.

    Breaking Changes to note:

    • Deserializing an empty quoted string ("") will not cause an error.
    • Deserializing an empty unquoted string will cause an error (it is read as an empty scalar).
    • NodeType_e enumeration values have changed due to the addition of KEYNIL and VALNIL.

    To check if a key or value is null, use the following methods available on NodeType, Tree, ConstNodeRef, and NodeRef:

    • .key_is_null()
    • .val_is_null()
    // Example of checking for nullity (conceptual)
if (node.key_is_null()) { /* ... */ }
if (node.val_is_null()) { /* ... */ }

  11. Create an empty Tree in Rapid YAML (v0.11.0 breaking change)

    master

    In version 0.11.0, a default-constructed Tree is now non-empty by default. Calling .root_id() on a default-constructed tree will return a valid ID because the tree is no longer empty.

    To create a truly empty tree (as in previous versions), you must use the capacity constructor with a capacity of zero: Tree tree(0);.

    Warning: You cannot call .root_id() on a tree created with zero capacity, as it has no root.

    // breaking change: default-constructed tree is now non-empty
Tree tree;
assert(!tree.empty()); // MODIFIED! was empty on previous version
id_type root = tree.root_id(); // OK. default-constructed tree is now non-empty

// to create an empty tree (as happened before):
Tree tree(0); // pass capacity of zero
assert(tree.empty()); // as expected

// but watchout this is no longer possible:
// id_type root = tree.root_id(); // ERROR: cannot get root of empty tree.

  12. Understand changes to root-level scalar parsing

    master

    As of version 0.2.1, the parsing behavior for root-level scalars has changed to improve compliance. Root-level scalars are now parsed into a DOCVAL instead of a SEQ->VAL.

    For example, a YAML document containing only a scalar:

    ---
this is a scalar
---

    will now be treated as a document value rather than a sequence containing a value.