serde-saphyr

repository·master·Indexed 41 days ago

https://github.com/bourumir-wyngs/serde-saphyr

A strongly typed YAML (de)serializer for Rust and Serde version 1.3.0. It emphasizes panic-free parsing, fast builds, and robust error reporting by parsing directly into target types without intermediate abstract values. Key features include support for properties, include tags, comment capture, and a Budget system to protect against DoS attacks. It includes a CLI for YAML validation and supports advanced configurations for duplicate key policies, indentation rules, and boolean strictness.

Tokens
28.3K
Snippets
71
Records
110
Agent score
54%

What's inside serde-saphyr

  1. Key features of serde-saphyr

    master

    Core Capabilities

    • Type-driven parsing: YAML is rejected early if it doesn't match expected Rust types.
    • Resource Management: Use Budget to enforce input limits (e.g., nesting depth or array size).
    • Tag Support: Use the Tagged<T> wrapper to capture and emit resolved YAML tags.
    • Comment Support: Use the Commented<T> wrapper to capture and emit comments.
    • Include Support: Optional !include support via custom or default resolvers.
    • Validation: Supports declarative validation via validator or garde crates, and advanced error reporting via miette.
    • Large Documents: The huge_documents feature allows handling YAML inputs far beyond 4 GiB by using a packed 48-bit internal representation for spans.
  2. Configure boolean parsing strictness

    master

    By default, if the target field is a boolean, serde-saphyr interprets standard YAML 1.1 boolean values (e.g., true, false, yes, no).

    To prevent incorrect inference (for example, when parsing into a JSON Value where you want literal strings), you can:

    1. Enclose the value in quotes in the YAML source.
    2. Set strict_booleans to true in your Options.
  3. Serialize with YAML anchors using Rc/Arc wrappers

    master

    To map Rust shared references (Rc, Arc) to YAML anchors and aliases, use the provided wrapper types. This allows for identity-preserving round-tripping where multiple fields in Rust point to the same memory location, and multiple fields in YAML use anchors (&a1) and aliases (*a1).

    Available wrappers:

    • RcAnchor<T> / ArcAnchor<T>: Emits an anchor on the first occurrence and aliases on subsequent occurrences.
    • RcWeakAnchor<T> / ArcWeakAnchor<T>: Serializes a weak reference; if the strong pointer is gone, it serializes as null.
    use serde::{Deserialize, Serialize};
    use serde_saphyr::RcAnchor;
    use std::rc::Rc;
    
    #[derive(Debug, Deserialize, Serialize)]
    struct Node {
        name: String,
    }
    
    #[derive(Debug, Deserialize, Serialize)]
    struct Document {
        primary: RcAnchor<Node>,
        alias: RcAnchor<Node>,
    }
    
    fn main() {
        let shared = Rc::new(Node { name: "shared node".to_string() });
        let document = Document {
            primary: RcAnchor::from(shared.clone()),
            alias: RcAnchor::from(shared),
        };
    
        let yaml = serde_saphyr::to_string(&document).expect("serialize anchors");
        // yaml will contain &a1 and *a1
    }
  4. Configure duplicate key handling policies

    master

    You can control how the parser handles duplicate keys in a mapping via Options. By default, duplicate keys result in an error.

    Available strategies:

    • First wins: Retains the first occurrence.
    • Last wins: Retains the last occurrence (requires buffering).

    Key behaviors for integer keys:

    • YAML integer keys (e.g., 0xB and 11) are compared by their numeric meaning (within i128/u128 range) regardless of the target Rust type.
    • If deserializing into a String, the original spelling is preserved (e.g., 0xB becomes "0xB").
    • Quoted keys or keys tagged !!str are treated as distinct from numeric keys.
    • Last wins buffering: When using LastWins with integer keys, the parser buffers the mapping to select the correct entry before passing it to Serde. Note that trailing comments and comments nested inside buffered values are not preserved in this mode.
  5. How Rust types act as a schema for deserialization

    master

    In serde-saphyr, the target Rust types serve as the explicit schema for parsing. This approach addresses the "Norway problem" by using the type system to resolve ambiguity:

    • Type-driven interpretation: If a field expects a bool, YAML shorthands like y, on, n, or off are interpreted as booleans. If it expects a String, those same values are treated as strings.
    • Numeric vs String: Hexadecimal notation (e.g., 0x2A) is parsed as an integer if the target is numeric, or as a string if the target is String.
    • Strictness: You can enforce stricter compatibility with other YAML parsers by setting no_schema: true in Options. This causes the parser to reject unquoted values that could be interpreted as something other than a string.
    • Validation: Because the schema is defined by Rust types, you can use crates like validator or garde to add field-level validation annotations directly to your structs.
  6. Understand fuzzing input formats

    master

    Fuzzing targets in this repository use different input strategies:

    • serialize: Uses the entire input as an untrimmed UTF-8-lossy string and attempts bounded recursive value generation. The corpus can contain readable strings, including whitespace and Unicode.
    • duplicate_keys and aliases_merges: Use leading control bytes to select a scenario; the remaining bytes are used to generate values.
    • document_streams: Uses five control bytes to define line endings, block style/chomping/indentation, content, blank lines, and separators, followed by up to 16 scalar selectors. Missing bytes default to zero.

    Note: For structured targets like document_streams, seed files are generator inputs rather than literal YAML fixtures.

  7. Protect against DoS using Budget configuration

    master

    To prevent denial-of-service (DoS) attacks from pathological YAML inputs that consume excessive time or memory, you can configure a Budget via Options.

    Key details:

    • What it tracks: Parser events, retained copies for anchor replaying, and property-interpolation depth/work.
    • Defaults: Intentionally permissive; tighten them for known input shapes or disable them if you only parse trusted, self-generated YAML.
    • Reader-based deserialization: Does not buffer the entire payload; it parses incrementally and enforces limits on bytes and structure.
    • Streaming: When using read_with_options to stream via an iterator, budget limits apply on a per-document basis rather than on the total input size.
    • Finding requirements: You can determine the necessary budget by using the library's web demo, running the library's main() executable with a YAML file path, or by using Options::with_budget_report to register a closure that fetches the budget programmatically.
  8. Enforce YAML indentation rules

    master

    To prevent semantic errors caused by incorrect spacing, you can enforce indentation rules during deserialization using the RequireIndent enum within your Options.

    Supported policies:

    • Require a consistent number of indentation columns throughout the document.
    • Ensure indentation is even.
    • Enforce that indentation is divisible by a specific number (e.g., 4 or 6).
  9. Use YAML merge keys for reducing redundancy

    master

    You can use YAML 1.1 merge keys (<<: *anchor) to reuse shared key-value pairs across multiple mappings. serde-saphyr supports this by default using MergeKeyPolicy::Merge, which expands both implicit << entries and explicit !!merge << entries.

    To control this behavior, use the merge_keys option in Options:

    • MergeKeyPolicy::Merge (default): Expands merge keys.
    • MergeKeyPolicy::AsOrdinary: Treats merge keys as regular mapping keys.
    • MergeKeyPolicy::Error: Rejects merge keys.
    // Example of using merge keys to inherit properties
    let yaml_input = r#"
    defaults: &defaults
      adapter: postgres
      host: localhost
    
    development:
      <<: *defaults
      database: dev_db
    
    production:
      <<: *defaults
      database: prod_db
    "#;
    
    let parsed: Config = serde_saphyr::from_str(yaml_input).expect("Failed to deserialize");
  10. Handle recursive YAML structures

    master

    To support recursive YAML structures (where a structure contains a reference to itself), you must use explicit wrappers to satisfy Rust's memory safety requirements:

    • Wrap the recursive data in RcRecursive<T> (or ArcRecursive<T>).
    • Use RcRecursion<T> (or ArcRecursion<T>) for any reference pointing to that data.
  11. Quickstart: Deserialize YAML into Rust types

    master

    To use serde-saphyr, define a Rust struct that implements serde::Deserialize and use the serde_saphyr::from_str function to parse a YAML string directly into your type. This approach is resource-efficient as it avoids constructing an intermediate abstract value tree.

    use serde::Deserialize;
    
    #[derive(Debug, Deserialize)]
    struct Config {
      name: String,
      enabled: bool,
      retries: i32,
    }
    
    fn main() {
        let yaml_input = r#"
          name: "My Application"
          enabled: true
          retries: 5
        ..."
        #;
    
        let config: Result<Config, _> = serde_saphyr::from_str(yaml_input);
    
        match config {
            Ok(parsed_config) => {
                println!("Parsed successfully: {:?}", parsed_config);
            }
            Err(e) => {
                eprintln!("Failed to parse YAML: {}", e);
            }
        }
    }
  12. Enable robotics domain extensions

    master

    The robotics feature allows parsing of YAML extensions used in robotics (like ROS), including mathematical expressions and angle conversions (e.g., deg(180), rad(pi), or 1 + 2*(3 - 4/5)).

    To use this capability, you must perform two steps:

    1. Enable the robotics feature in your Cargo.toml.
    2. Set angle_conversions: true in the serde_saphyr::options! macro.

    Note: This is a simple expression calculator implemented in Rust, not a full language interpreter. Safety hardening is included to limit expression depth, digit counts, and fraction precision.

    use serde::Deserialize;
    
    #[derive(Debug, Deserialize)]
    struct RoboFloats {
        func_deg: f64,
        func_rad: f64,
    }
    
    fn main() {
        let yaml = "func_deg: deg(180)\nfunc_rad: rad(pi)\n";
        let options = serde_saphyr::options! {
            angle_conversions: true,
        };
    
        let v: RoboFloats = serde_saphyr::from_str_with_options(yaml, options)
            .expect("parse robotics YAML");
        
        assert!((v.func_deg - std::f64::consts::PI).abs() < 1e-12);
        assert!((v.func_rad - std::f64::consts::PI).abs() < 1e-12);
    }