defusedxml

repository·main·Indexed 20 days ago

https://github.com/tiran/defusedxml

A Python library designed to protect applications from XML-based attacks, including 'Billion Laughs', quadratic blowup, and external entity expansion (XXE). It provides secure alternatives to standard library XML parsers through modules such as defusedxml.ElementTree, defusedxml.minidom, defusedxml.sax, defusedxml.pulldom, and a monkey patch for xmlrpc.

Tokens
3.4K
Snippets
8
Records
19
Agent score
69%

What's inside defusedxml

  1. Understand XML attack vectors

    main

    XML parsers can be vulnerable to several types of attacks that exploit less common XML features like entity expansions and DTDs. Understanding these is critical for securing your application:

    • Billion Laughs (Exponential Entity Expansion): Uses nested entities to expand a small XML file into gigabytes of memory, causing DoS via memory exhaustion and high CPU load.
    • Quadratic Blowup: Repeats a single large entity many times. It avoids nested-depth countermeasures but still causes massive memory consumption.
    • External Entity Expansion (Remote): Uses URIs (e.g., http://) in entity declarations to force the parser to download remote resources. This can be used for SSRF (Server-Side Request Forgery), bypassing firewalls, or DoS.
    • External Entity Expansion (Local File): Uses file:// or relative paths to force the parser to read local files (e.g., /etc/passwd), potentially leaking sensitive configuration or system data.
    • DTD Retrieval: Some libraries automatically retrieve Document Type Definitions from remote or local locations, leading to similar risks as external entity expansion.
  2. Prevent XPath injection attacks in lxml

    main

    When using lxml, avoid using Python string formatting to build XPath queries, as this makes your application vulnerable to XPath injection (similar to SQL injection). Instead, use parameterized XPath queries provided by the .xpath() method. This ensures that arguments are properly quoted and validated.

    # DON'T
    >>> tree.xpath("/tag[@id='%s']" % value)
    
    # instead do
    >>> tree.xpath("/tag[@id=$tagid]", tagid=name)
  3. How to check for built-in XML mitigations

    main

    You can verify if your Python environment has built-in mitigations against XML bombs by checking the Python version and the libexpat version via pyexpat.

    Mitigations are generally present if:

    1. Python version is $\ge$ 3.7.1, 3.8.12, 3.9.7, or 3.10.0 (August 2021).
    2. libexpat version is $\ge$ 2.4.0.
    import sys
    import pyexpat
    
    has_mitigations = (
        sys.version_info >= (3, 7, 1) and
        pyexpat.version_info >= (2, 4, 0)
    )
  4. Use defusedxml to prevent XML vulnerabilities

    main

    To protect your application from XML-based attacks (like Billion Laughs, quadratic blowup, or external entity expansion), replace your standard library XML imports with the corresponding defusedxml modules.

    Note: defusedxml modules are not drop-in replacements for all features. They focus on secure parsing and loading. For other XML operations (like creating elements or converting to strings), use the original standard library classes/functions in conjunction with the objects returned by defusedxml.

    # Instead of:
    # from xml.etree.ElementTree import parse
    # et = parse(xmlfile)
    
    # Use:
    from defusedxml.ElementTree import parse
    et = parse(xmlfile)
  5. Use defusedxml to protect XML parsing

    main

    To protect your application from XML vulnerabilities (like XML bombs), replace standard library XML imports with the corresponding defusedxml modules.

    Important: defusedxml modules are not drop-in replacements for the entire standard library. They only provide functions and classes for parsing and loading. For all other operations (like manipulating elements, creating new elements, or converting to strings), use the original xml.etree.ElementTree (or other stdlib) classes and functions.

    Example of correct usage:

    from defusedxml import ElementTree as DET
    from xml.etree.ElementTree as ET
    
    # Use defusedxml for parsing
    root = DET.fromstring("<root/>")
    
    # Use stdlib for manipulation and serialization
    root.append(ET.Element("item"))
    print(ET.tostring(root))
    from defusedxml import ElementTree as DET
    from xml.etree.ElementTree as ET
    
    root = DET.fromstring("<root/>")
    root.append(ET.Element("item"))
    ET.tostring(root)
  6. Secure XML parsing in .NET / C#

    main

    To prevent XML Denial of Service (DoS) attacks in .NET, configure XmlReaderSettings to prohibit DTDs, limit entity expansion, and nullify the XmlResolver.

    XmlReaderSettings settings = new XmlReaderSettings();
    settings.ProhibitDtd = false;
    settings.MaxCharactersFromEntities = 1024;
    settings.XmlResolver = null;
    XmlReader reader = XmlReader.Create(stream, settings);
  7. Secure XML parsing in Java (Xerces)

    main

    To secure a DocumentBuilderFactory in Java against billion laughs attacks and other entity-related exploits, disable XInclude, disable entity reference expansion, and enable FEATURE_SECURE_PROCESSING. You should also explicitly disallow DOCTYPE declarations or disable external general/parameter entities.

    DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
    builderFactory.setXIncludeAware(False);
    builderFactory.setExpandEntityReferences(False);
    builderFactory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, True);
    # or if you need DTDs
    builderFactory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", True);
    # or if you need DTDs
    builderFactory.setFeature("http://xml.org/sax/features/external-general-entities", False);
    builderFactory.setFeature("http://xml.org/sax/features/external-parameter-entities", False);
    builderFactory.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", False);
    builderFactory.setFeature("http://apache.org/xml/features/nonvalidating/load-dtd-grammar", False);
  8. Secure XML parsing in Ruby (REXML)

    main

    To protect Ruby's REXML parser from entity expansion attacks (both quadratic and exponential), you must manually disable the feature by setting the entity_expansion_limit to 0.

    REXML::Document.entity_expansion_limit = 0
  9. Secure xmlrpc with defusedxml.xmlrpc

    main

    The defusedxml.xmlrpc module provides a monkey patch for the standard library's xmlrpc package (Python 3.x) or xmlrpclib (Python 2.x). This patch protects against XML attacks, decompression bombs, and excessively large requests/responses.

    • Default Limit: 30 MB for requests, responses, and gzip decompression.
    • Customizing Limits: Modify the MAX_DATA module variable. Setting it to -1 disables the limit.
    • Control: Use monkey_patch() to enable fixes and unmonkey_patch() to revert to the original state.
  10. Configure defusedxml.xmlrpc protection

    main

    The defusedxml.xmlrpc module uses a monkey patch to protect against XML attacks and decompression bombs.

    • Use monkey_patch() to enable protection.
    • Use unmonkey_patch() to revert to the original state.
    • The default limit for requests, responses, and gzip decompression is 30 MB.
    • You can modify this limit by changing the MAX_DATA module variable. Setting MAX_DATA = -1 disables the limit.
  11. Secure lxml usage with defusedxml.lxml

    main

    While lxml is inherently safer than many parsers (it has built-in mitigations for Billion Laughs and quadratic blowup), the defusedxml.lxml module provides an example of how to wrap it securely.

    Note: This module is deprecated and will be removed in a future release.

    To manually secure lxml without this module, you can use a custom parser object with resolve_entities=False:

    from lxml import etree
    
    parser = etree.XMLParser(resolve_entities=False)
    root = etree.fromstring("<example/>", parser=parser)
  12. Available defusedxml modules and APIs

    main

    The defusedxml package provides specialized modules for different XML parsing interfaces. Use the module that matches your existing workflow with the standard library.

    • defusedxml.ElementTree: Provides parse(), iterparse(), fromstring(), and XMLParser. (Note: defusedxml.cElementTree is deprecated; use this instead).
    • defusedxml.minidom: Provides parse() and parseString().
    • defusedxml.sax: Provides parse(), parseString(), and make_parser().
    • defusedxml.pulldom: Provides parse() and parseString().
    • defusedxml.expatreader: Provides create_parser() and DefusedExpatParser.
    • defusedxml.expatbuilder: Provides parse(), parseString(), DefusedExpatBuilder, and DefusedExpatBuilderNS.
    • defusedxml.xmlrpc: Implements a monkey patch for the stdlib's xmlrpc (3.x) or xmlrpclib (2.x) to protect against XML attacks and decompression bombs.