roxmltree

repository·master·Indexed 19 days ago

https://github.com/razrfalcon/roxmltree

A high-performance, read-only XML tree parser for Rust. Designed for efficient data retrieval with zero dependencies, zero unsafe code, and minimal memory overhead. It supports XML namespace resolution, DTD ENTITY resolution, and provides a variety of tree traversal methods via the Document and Node APIs. Memory usage can be further optimized by disabling the positions feature.

Tokens
3.8K
Snippets
12
Records
26
Agent score
68%

What's inside roxmltree

  1. Core concepts of roxmltree

    master

    roxmltree represents an XML document as a read-only tree. This design choice allows for significant optimizations when the primary goal is data retrieval rather than document modification.

    Key characteristics include:

    • Read-only: The tree cannot be modified after parsing.
    • Position Preserving: By default, roxmltree keeps all node and attribute positions from the original document, allowing you to retrieve them easily.
    • Zero Unsafe: The library forbids unsafe code and is designed to never panic.
    • Zero Dependencies: It has no external dependencies.
    • Parsing Behavior: It aims to mimic the behavior of Python's lxml.
  2. How CDATA and Text are handled

    master

    roxmltree simplifies text representation by merging CDATA sections and resolving entities into standard text nodes.

    • CDATA: Content within <![CDATA[...]]> is embedded into the surrounding text node.
    • Text: All entity references (like &amp; or custom entities) are unescaped and resolved into their literal values.
    <!-- CDATA Example -->
    <p>t<![CDATA[e&#x20;]]>&#x20;x<![CDATA[t]]></p>
    <!-- Parsed as: -->
    <p>te&#x20; xt</p>
    
    <!-- Entity Example -->
    <!DOCTYPE test [
        <!ENTITY b 'Some&#x20;text'>
    ]>
    <p>&b;</p>
    <!-- Parsed as: -->
    <p>Some text</p>
  3. Understand roxmltree's XML parsing strategy

    master

    roxmltree uses a specific parsing strategy that makes decisions about which XML components to preserve, ignore, or transform. Understanding these behaviors is critical for predicting how your XML document will be represented in the tree.

    Key behaviors include:

    • XML Declarations: Completely ignored. The version must be 1.* or an error occurs. encoding is ignored as only UTF-8 is supported.
    • DTD: Only ENTITY objects are resolved; all other DTD elements are ignored.
    • Comments & Processing Instructions: All are preserved in the tree.
    • Whitespaces: All whitespaces inside the root element are preserved, including escaped characters.
    • CDATA: CDATA sections are merged/embedded directly into text nodes.
    • Text: All text is unescaped and entity references are resolved.
    • Attributes: Follows standard XML Attribute-Value Normalization.
    • Namespaces: Full support for XML namespace resolution is provided.
  4. How DTD ENTITY resolution works in roxmltree

    master

    roxmltree only resolves ENTITY objects within a DTD. Other DTD declarations are ignored. When an entity is resolved, it is parsed as part of the document structure (e.g., an entity containing XML tags will result in actual element nodes, not just raw text).

    ```xml
    <!DOCTYPE test [
        <!ENTITY a 'text<p/>text'>
    ]>
    <e>&a;</e>

    Parsed result:

    <e>text<p/>text</e>

    (Note: <p/> is treated as an element node, not text.)

  5. Optimize memory usage by disabling positions

    master

    To reduce memory overhead, you can disable the positions feature. This shaves 8 bytes from each node and attribute.

    Note that memory usage is driven by the number of nodes and attributes rather than file size. On average, memory overhead is approximately 6-8x the file size. For example, a 1.1GB XML file peaks at ~7.6GB RAM with default features, but drops to ~6.8GB RAM when positions is disabled.

  6. Traverse the XML tree

    master

    You can traverse the document using various methods:

    • descendants(): Returns an iterator over all descendant nodes in the document (shorthand for doc.root().descendants()).
    • children(): (Available on Node) Returns an iterator over the immediate children of a node.
    • first_child(): (Available on Node) Returns the first child of a node.

    Nodes follow document-order, meaning they are ordered as they appear in the source text (similar to XPath and DOM selectors).

  7. Parse XML and find elements with roxmltree

    master

    Use roxmltree::Document::parse to create a read-only tree from an XML string. Once parsed, you can traverse the tree using iterators like descendants() to find specific nodes based on attributes or tag names.

    // Find element by id.
    let doc = roxmltree::Document::parse("<rect id='rect1'/>")?;
    let elem = doc.descendants().find(|n| n.attribute("id") == Some("rect1"))?;
    assert!(elem.has_tag_name("rect"));
  8. Troubleshoot XML parsing errors

    master

    When parsing XML with roxmltree, you may encounter several error types if the input is malformed or contains undefined references. Understanding these errors helps in identifying the exact location and cause of the failure in your XML source.

    Common error variants include:

    • UnknownEntityReference(String, Position): An entity reference (e.g., &name;) was found, but the name is not defined in the document's entities.
    • MalformedEntityReference(Position): An ampersand & was found that does not start a valid character or entity reference.
    • InvalidAttributeValue(Position): An invalid character (such as an escaped <) was found within an attribute value.
    • UnknownNamespace(String, Position): A namespace prefix was used (e.g., prefix:name), but the prefix is not defined in the document's namespace declarations.
  9. Access an element's tail text

    master

    In XML, 'tail text' is the text that follows an element's closing tag but precedes the next sibling. You can retrieve this text using the tail() method on an element node.

    let doc = roxmltree::Document::parse("
    <root>
        text1
        <p/>
        text2
    </root>
    ").unwrap();
    
    let p = doc.descendants().find(|n| n.has_tag_name("p")).unwrap();
    assert_eq!(p.tail(), Some("\n    text2\n"));
  10. Query element attributes and namespaces

    master

    For Element nodes, you can access attributes and namespaces:

    • attribute(name): Returns the value of an attribute as a &str. The name can be a simple string or an ExpandedName (for namespaced attributes).
    • has_attribute(name): Checks if an attribute exists.
    • attributes(): Returns an iterator over all Attribute objects.
    • namespaces(): Returns an iterator over the namespaces declared for the element.
    • lookup_prefix(uri): Finds the prefix associated with a given namespace URI.
    • lookup_namespace_uri(prefix): Finds the URI associated with a given prefix.
    // Accessing a namespaced attribute
    let val = node.attribute(("http://www.w3.org", "a"));
    
    // Accessing a local attribute
    let val = node.attribute("a");
  11. Configure parsing with ParsingOptions

    master

    Use Document::parse_with_options to customize the parsing behavior.

    Options

    • allow_dtd: (bool) Whether to allow DTD parsing. Set to false (default) for security to prevent DTD-based attacks. If false, an XML document containing a DTD will return Error::DtdDetected.
    • nodes_limit: (u32) The maximum number of nodes to parse. Useful for limiting memory usage when processing untrusted input. Defaults to u32::MAX (no limit).
    • entity_resolver: (Option<&EntityResolver>) A custom function to resolve external entities.
    let opt = roxmltree::ParsingOptions {
        allow_dtd: true,
        nodes_limit: 1000,
        entity_resolver: None,
    };
    let doc = roxmltree::Document::parse_with_options("<e/>", opt).unwrap();