yaml-rust2

repository·master·Indexed 42 days ago

https://github.com/ethiraric/yaml-rust2

A fully YAML 1.2 compliant YAML parser and serializer written in pure Rust. Designed as a memory-safe successor to the original yaml-rust crate, version 0.13.0 provides tools for parsing YAML strings via YamlLoader, accessing data through indexing and type conversion, and serializing objects using YamlEmitter. It includes development tools such as bench_compare for performance testing, dump_events for debugging, and gen_large_yaml for stress-testing.

Tokens
7.4K
Snippets
29
Records
35
Agent score
51%

What's inside yaml-rust2

  1. Available `yaml-rust2` development tools

    master

    The yaml-rust2 repository includes several development tools. Note that due to dependency management, only some of these are available as binaries directly from the yaml-rust2 crate. Use the following invocation patterns to run them:

    • bench_compare: Run via cargo bench_compare.
    • dump_events: Run via cargo run --bin dump_events -- [...].
    • run_bench: Run via cargo run --bin run_bench -- [...].
    • time_parse: Run via cargo run --bin time_parse -- [...].
    | Tool | Invocation |
    |------|------------|
    | `bench_compare` | `cargo bench_compare` |
    | `dump_events` | `cargo run --bin dump_events -- [...]` |
    | `run_bench` | `cargo run --bin run_bench -- [...]` |
    | `time_parse` | `cargo run --bin time_parse -- [...]` |
  2. Access YAML data using Indexing and Type conversion

    master

    The Yaml type implements Index for both usize (arrays) and &str (maps).

    • doc[index] (where index is usize) assumes the container is an array.
    • doc["key"] (where key is &str) assumes the container is a map with string keys.

    If the access pattern does not match the actual structure (e.g., accessing a key on an array), the library returns Yaml::BadValue instead of panicking. For non-standard structures (like maps with complex type keys), use the as_XXX family of methods (e.g., as_str(), as_f64(), as_bool()) to safely access values.

    // Index access
    assert_eq!(doc["foo"][0].as_str().unwrap(), "list1");
    
    // Safe error handling for invalid access
    if doc["INVALID_KEY"].is_badvalue() {
        // handle error
    }
  3. Switch from yaml-rust to yaml-rust2

    master

    To migrate from the unmaintained yaml-rust crate to yaml-rust2, you can use one of three methods depending on how much code you want to change.

    Update your Cargo.toml and change all occurrences of yaml_rust to yaml_rust2 in your source code.

    Method 2: Use a Rust Alias

    Update your Cargo.toml to use yaml-rust2 but alias it as yaml_rust in your code using use yaml_rust2 as yaml_rust;.

    Method 3: Cargo Package Alias (Zero Code Changes)

    You can tell Cargo to treat yaml-rust2 as the provider for the yaml-rust dependency. This allows you to keep using use yaml_rust::... in your code without any modifications.

    # Method 1: Direct replacement
    -yaml-rust = "0.4.4"
    +yaml-rust2 = "0.8.0"
    
    # Method 3: Cargo alias (No code changes required)
    -yaml-rust = "0.4.4"
    +yaml-rust = { version = "0.6", package = "yaml-rust2" }
  4. Use bench_compare to compare YAML parser performance

    master

    The bench_compare tool is used to compare the time different YAML parsers take to process the same input files. It operates by executing one of two required binaries (time_parse or run_bench) for each parser defined in a configuration file.

    Usage Syntax:

    bench_compare time_parse
    bench_compare run_bench

    To run these commands, you must have a bench_compare.toml file in your current directory containing the parser definitions and output paths.

    bench_compare time_parse
    bench_compare run_bench
  5. Debug parser events with `dump_events`

    master

    The dump_events tool is a debugging helper that outputs the sequence of events emitted by the parser for a specific YAML file.

    To get an in-depth overview of the scanner and parser steps (including parser states and markers), set the YAMLRUST2_DEBUG=1 environment variable.

    Example usage:

    cargo run --bin dump_events -- input.yaml
    cargo run --bin dump_events -- input.yaml
  6. Upgrade from yaml-rust as a drop-in replacement

    master

    You can replace the original yaml-rust crate with yaml-rust2 without changing your source code by using the package key in your Cargo.toml. This allows you to continue using use yaml_rust::{...} in your modules while actually using the yaml-rust2 implementation.

    [dependencies]
    yaml-rust = { version = "#.#", package = "yaml-rust2" }
  7. Configure bench_compare via bench_compare.toml

    master

    The bench_compare tool requires a bench_compare.toml file in the current directory. All fields must be explicitly set; default values are not supported.

    Required Configuration Fields:

    • yaml_input_dir: Path to the directory containing the input YAML files.
    • iterations: Number of iterations to perform (used when running run_bench).
    • yaml_output_dir: Directory where run_bench YAML outputs are saved.
    • csv_output: Path to the CSV file that aggregates times for each parser and file.
    • [[parsers]]: An array of tables defining each parser. Each parser block requires:
      • name: The name of the parser (used for logging).
      • path: The directory containing the parser's run_bench and time_parse binaries.
    yaml_input_dir = "bench_yaml"
    iterations = 10
    yaml_output_dir = "yaml_output"
    csv_output = "benchmark.csv"
    
    [[parsers]]
    name = "yaml-rust2"
    path = "target/release/"
    
    # [[parsers]]
    # name = "libfyaml"
    # path = "../libfyaml/build"
  8. Manipulate YAML documents with the `Yaml` enum

    master

    The Yaml enum is the primary way to interact with parsed YAML data. It represents various YAML types including integers, booleans, strings, arrays, hashes, and nulls.

    Key features:

    • Type Conversion: Use as_i64(), as_bool(), as_str(), as_vec(), and as_hash() to attempt to access the underlying data. For floating point numbers, use as_f64() which parses the Real variant on demand.
    • Safe Access: If you attempt to access a non-existent key in a hash or an out-of-bounds index in an array, the library returns Yaml::BadValue instead of panicking (when using the Index trait).
    • Fallback Values: Use .or(other) or .borrowed_or(&other) to provide a default value if the current node is Yaml::Null or Yaml::BadValue.
    use yaml_rust2::Yaml;
    
    let foo = Yaml::from_str("-123");
    assert_eq!(foo.as_i64().unwrap(), -123);
    
    // Use .or() to handle nulls or missing values
    let val = Yaml::Null.or(Yaml::Integer(42));
    assert_eq!(val.as_i64().unwrap(), 42);
  9. Use the low-level event-based parsing API

    master

    The yaml-rust2 parser provides a low-level, push-based (event-driven) API via the EventReceiver trait. Instead of loading a full document into memory, you implement this trait to react to individual YAML events as they are emitted.

    Event Hierarchy

    1. The stream starts with Event::StreamStart.
    2. Followed by Event::DocumentStart.
    3. Then the document content (e.g., Event::MappingStart, Event::SequenceStart, or Event::Scalar).
    4. The stream ends with Event::StreamEnd.

    Implementation Patterns

    • EventReceiver: Implement this for a simple event stream. It provides on_event(ev: Event).
    • MarkedEventReceiver: Implement this if you need to know the location (line/column) of each event. It provides on_event(ev: Event, mark: Marker). Note that all EventReceiver implementations automatically implement MarkedEventReceiver.
    # use yaml_rust2::parser::{Event, EventReceiver, Parser};
    
    /// Sink of events. Collects them into an array.
    struct EventSink {
        events: Vec<Event>,
    }
    
    /// Implement `on_event`, pushing into `self.events`.
    impl EventReceiver for EventSink {
        fn on_event(&mut self, ev: Event) {
            self.events.push(ev);
        }
    }
    
    /// Load events from a yaml string.
    fn str_to_events(yaml: &str) -> Vec<Event> {
        let mut sink = EventSink { events: Vec::new() };
        let mut parser = Parser::new_from_str(yaml);
        // Load events using our sink as the receiver.
        parser.load(&mut sink, true).unwrap();
        sink.events
    }
  10. Decode YAML with custom encoding error handling using `YamlDecoder`

    master

    When reading raw bytes (e.g., from a file or network), use YamlDecoder to handle character encoding (UTF-8, UTF-16, etc.) and potential malformations.

    Decoding Traps (YAMLDecodingTrap):

    • Strict: Returns a LoadError::Decode on malformed sequences.
    • Ignore: Skips the offending bytes.
    • Replace: Replaces malformed bytes with the Unicode replacement character (�).
    • Call(fn): Executes a user-provided callback function to handle the error manually.

    Example usage:

    use yaml_rust2::yaml::{YamlDecoder, YAMLDecodingTrap};
    
    let bytes = b"a\xa9: 1"; // Contains invalid UTF-8
    let mut decoder = YamlDecoder::read(bytes)
        .encoding_trap(YAMLDecodingTrap::Ignore);
    
    let docs = decoder.decode().unwrap();
    use yaml_rust2::yaml::{YamlDecoder, YAMLDecodingTrap};
    
    let bytes = b"---\na\xa9: 1\nb: 2.2\nc: [1, 2]\n";
    let out = YamlDecoder::read(bytes)
        .encoding_trap(YAMLDecodingTrap::Ignore)
        .decode()
        .unwrap();