quick-xml

repository·master·Indexed 23 days ago

https://github.com/tafia/quick-xml

A high-performance, almost zero-copy XML pull reader and writer for Rust. It features memory-efficient parsing via a Reader API, a streaming Writer API for generating XML, and optional Serde integration for mapping XML structures to Rust structs. Supports various encodings and custom entity resolution through the EntityResolver trait.

Tokens
12.2K
Snippets
18
Records
53
Agent score
81%

What's inside quick-xml

  1. How quick-xml works: Reader, Writer, and Serde

    master

    The quick-xml library provides three primary ways to interact with XML:

    1. Reader (Pull Parsing): A low-level, high-performance streaming API. It uses an event-based model where you pull Events (like Start, End, Text, Eof) from a buffer. It is designed to be zero-copy and memory-efficient by allowing buffer reuse.
    2. Writer: A streaming API for constructing XML. It works by writing Events to an underlying std::io::Write sink. It is often used in conjunction with a Reader to transform XML on the fly.
    3. Serde Integration: A high-level abstraction that maps XML structures to Rust data types. This is built on top of the core parsing logic and is ideal when you want to work with strongly-typed data rather than raw events.
  2. Run fuzzing with cargo-fuzz

    master

    To run fuzzing targets for quick-xml, use cargo fuzz run. It is recommended to use the -O flag to enable optimizations; this helps avoid false positives triggered by debug_assert! statements during the fuzzing process.

    Example command:

    cargo fuzz run -O -j4 fuzz_target_1
  3. Write XML using the Writer API

    master

    The Writer allows you to generate XML by writing Events. You can wrap any type that implements std::io::Write (like Vec<u8> via Cursor or a file).

    When transforming XML, you can:

    1. Read an event from a Reader.
    2. Create new elements using BytesStart::new("name").
    3. Copy or extend attributes from existing events using elem.extend_attributes().
    4. Add new attributes using elem.push_attribute(("key", "value")).
    5. Write the event to the Writer using writer.write_event(event).

    To retrieve the written data, use writer.into_inner().

    use quick_xml::events::{Event, BytesEnd, BytesStart};
    use quick_xml::reader::Reader;
    use quick_xml::writer::Writer;
    use std::io::Cursor;
    
    let xml = r#"<this_tag k1=\"v1\" k2=\"v2\"><child>text</child></this_tag>"#;
    let mut reader = Reader::from_str(xml);
    reader.config_mut().trim_text(true);
    let mut writer = Writer::new(Cursor::new(Vec::new()));
    loop {
        match reader.read_event() {
            Ok(Event::Start(e)) if e.name().as_ref() == b"this_tag" => {
                let mut elem = BytesStart::new("my_elem");
                elem.extend_attributes(e.attributes().map(|attr| attr.unwrap()));
                elem.push_attribute(("my-key", "some value"));
                assert!(writer.write_event(Event::Start(elem)).is_ok());
            },
            Ok(Event::End(e)) if e.name().as_ref() == b"this_tag" => {
                assert!(writer.write_event(Event::End(BytesEnd::new("my_elem"))).is_ok());
            },
            Ok(Event::Eof) => break,
            Ok(e) => assert!(writer.write_event(e).is_ok()),
            Err(e) => panic!("Error at position {}: {:?}", reader.error_position(), e),
        }
    }
    
    let result = writer.into_inner().into_inner();
    let expected = r#"<my_elem k1=\"v1\" k2=\"v2\" my-key=\"some value\\"><child>text</child></my_elem>"#;
    assert_eq!(result, expected.as_bytes());
  4. Run the XML libraries benchmark suite

    master

    The compare package is a standalone project used to benchmark different XML parser implementations. To execute the benchmarks, navigate to the compare directory from the quick_xml root and use cargo bench.

    cd compare
    cargo bench
  5. Read XML using the Reader API

    master

    The Reader provides a high-performance, almost zero-copy way to parse XML. It uses Cow to minimize allocations and allows for buffer reuse to keep memory usage low.

    Because the Reader outputs borrowed data, it does not implement Iterator. Instead, you should use a loop with read_event_into(&mut buf) to process events. To minimize memory overhead, call buf.clear() at the end of each loop iteration if you are not keeping a borrow of the buffer elsewhere.

    Key configuration:

    • Use reader.config_mut().trim_text(true) to ignore whitespace between tags.
    use quick_xml::events::Event;
    use quick_xml::reader::Reader;
    
    let xml = r#"<tag1 att1 = \"test\">\n                <tag2><!--Test comment-->Test</tag2>\n                <tag2>Test 2</tag2>\n             </tag1>"#;
    let mut reader = Reader::from_str(xml);
    reader.config_mut().trim_text(true);
    
    let mut count = 0;
    let mut txt = Vec::new();
    let mut buf = Vec::new();
    
    loop {
        match reader.read_event_into(&mut buf) {
            Err(e) => panic!("Error at position {}: {:?}", reader.error_position(), e),
            Ok(Event::Eof) => break,
    
            Ok(Event::Start(e)) => {
                match e.name().as_ref() {
                    b"tag1" => println!("attributes values: {:?}",
                                        e.attributes().map(|a| a.unwrap().value)
                                        .collect::<Vec<_>>()),
                    b"tag2" => count += 1,
                    _ => (),
                }
            }
            Ok(Event::Text(e)) => txt.push(e.decode().unwrap().into_owned()),
    
            _ => (),
        }
        buf.clear();
    }
  6. Serialization of XML Attributes

    master

    When serializing data intended for XML attributes:

    • Structs: Fields are serialized as key-value pairs within the opening tag (e.g., <Attributes key="answer" val="42 42"/>).
    • Unsupported Maps: Serializing map types into attributes is not supported. This applies to both standard attribute maps and mixed maps containing both attribute keys (starting with @) and regular keys.
  7. How quick-xml modes of operation work

    master

    quick-xml provides two primary ways to interact with XML data:

    1. Streaming API (StAX model): Best for large XML documents that cannot fit entirely in memory. This is a pull-based model where you explicitly request the next XML event (similar to a database cursor). This is implemented via the Reader and Writer structs.
    2. Serde Support: If the serialize feature is enabled, you can use Serde to directly serialize and deserialize Rust structs to/from XML, bypassing manual event handling. This is implemented in the de and se modules.

    Additionally, if the async-tokio feature is enabled, quick-xml supports asynchronous reading and writing using tokio.

  8. Configure the XML Reader with Config

    master

    The Config struct allows you to customize the behavior of the Reader. You can access the current configuration using Reader::config() and modify it using Reader::config_mut().

    Key configuration options include:

    • allow_dangling_amp: If true, lone ampersands (without a paired semicolon) are allowed in text. Default is false.
    • allow_unmatched_ends: If true, unmatched closing tags are permitted. Default is false.
    • check_comments: If true, validates that comments do not contain --. Default is false.
    • check_end_names: If true, detects mismatched closing tag names. Default is true.
    • expand_empty_elements: If true, splits <tag/> into a Start event followed by an End event. Default is false.
    • trim_markup_names_in_closing_tags: If true, trims trailing whitespace in closing tags like </a >. Default is true.
    • trim_text_start / trim_text_end: Trims leading/trailing whitespace in Text events. Default is false.

    Warning: Using trim_text_start or trim_text_end can lead to incorrect behavior for text delimited by comments, processing instructions, or CDATA sections. For precise control, use BytesText::inplace_trim_start or BytesText::inplace_trim_end manually.

    let mut reader = Reader::from_str("text with & &amp; & alone");
    reader.config_mut().allow_dangling_amp = true;
  9. Serialization behavior for `$value` fields

    master

    When targeting a $value field (typically used within enum variants to represent nested XML elements), the following rules apply:

    • Primitives: Serialized as plain text (e.g., 42u8 -> "42").
    • Strings and Characters: Serialized with XML escaping.
    • Options: Option::None serializes to an empty string. Option::Some(val) serializes the inner value.
    • Enums:
      • Unit variants: Serialized as self-closing tags (e.g., <Unit/>).
      • Newtype variants: The inner value is serialized, and the variant name is used as the tag name (e.g., Newtype(42) -> <Newtype>42</Newtype>).
      • Tuple variants: Each element in the tuple is serialized as its own element using the variant name as the tag (e.g., Tuple("first", 42) -> <Tuple>first</Tuple><Tuple>42</Tuple>).
    • Structs: Serialized as nested XML elements using the struct/variant name as the tag.
    • Unsupported Types:
      • Sequences/Tuples of Primitives: Serializing a sequence of primitives without delimiters is unsupported because it cannot be reliably deserialized back.
      • Maps: Serialization of map types is not supported in $value fields.
      • Bytes: serialize_bytes is currently unsupported.
  10. Handle BOM (Byte Order Mark) in XML readers

    master

    The Reader handles the Byte Order Mark (BOM) automatically based on the enabled features:

    • When encoding feature is enabled: The reader detects the encoding from the BOM (e.g., UTF-8) and strips the BOM character from the stream.
    • When encoding feature is disabled: The reader assumes UTF-8 and strips the BOM character for consistency.

    If a BOM is present, the first read_event call will typically return the content following the BOM, or if the BOM is the only content, it may be treated as text or EOF depending on the specific sequence.

  11. How TextDeserializer handles XML text content

    master

    The TextDeserializer is used to deserialize a single text node from a mixed sequence of tags and text. It interprets the raw textual content of an XML element according to specific rules based on the target Rust type:

    • Numbers: Parsed from text using FromStr.
    • Booleans: Converted according to XML specification:
      • "true" and "1" $\rightarrow$ true
      • "false" and "0" $\rightarrow$ false
      • Other values trigger string/borrowed string visitor methods.
    • Strings: Returned as-is.
    • Characters: Returned as strings; an error is returned if the content is empty or contains more than one character.
    • Option: Empty text is deserialized as None; non-empty text is deserialized as Some using the same rules.
    • Units (()) and Unit Structs: Always succeed; the content is ignored.
    • Newtype Structs: Forwards deserialization to the inner type.
    • Sequences, Tuples, and Tuple Structs: Deserialized using SimpleTypeDeserializer (the text content is treated as the sequence data).
    • Structs and Maps: Calls string visitor methods; the type is responsible for handling the string data.
    • Enums:
      • The variant name is deserialized from the $text key.
      • Unit variants: Returns ().
      • Newtype variants: Forwards deserialization to the inner type.
      • Tuple and Struct variants: Deserialized using SimpleTypeDeserializer.