htmlparser2

repository·master·Indexed 26 days ago

https://github.com/fb55/htmlparser2

A fast and forgiving HTML/XML parser (version 12.0.0) designed for high performance with minimal allocations. It provides a low-level callback interface via the Parser class, a WritableStream for streaming input, and a higher-level DOM-based API using parseDocument and DomUtils for searching, modifying, and serializing DOM trees. It also includes a parseFeed method for processing RSS, RDF, and Atom feeds.

Tokens
2.8K
Snippets
8
Records
16
Agent score
39%

What's inside htmlparser2

  1. Search the DOM with DomUtils

    master

    Use DomUtils (re-exported by htmlparser2) to find nodes within a DOM tree. You can find elements by ID, tag name, class, or via custom test functions. For CSS selector queries, it is recommended to use the css-select package.

    import * as htmlparser2 from "htmlparser2";
    
    const dom = htmlparser2.parseDocument(`<div><p id="greeting">Hello</p></div>`);
    
    // Find elements by ID, tag name, or class
    const greeting = htmlparser2.DomUtils.getElementById("greeting", dom);
    const paragraphs = htmlparser2.DomUtils.getElementsByTagName("p", dom);
    
    // Find elements with custom test functions
    const all = htmlparser2.DomUtils.findAll(
        (el) => el.attribs?.class === "active",
        dom,
    );
    
    // Get text content
    htmlparser2.DomUtils.textContent(greeting); // "Hello"
  2. Modify and serialize the DOM

    master

    You can modify the DOM tree using DomUtils methods like removeElement, appendChild, prependChild, append, prepend, and replaceElement. To convert the DOM back into an HTML string, use DomUtils.getOuterHTML(dom) (which uses dom-serializer internally).

    import * as htmlparser2 from "htmlparser2";
    
    const dom = htmlparser2.parseDocument(
        `<ul><li class="apple">Apple</li><li class="orange">Orange</li></ul>`,
    );
    
    // Remove the first <li>
    const items = htmlparser2.DomUtils.getElementsByTagName("li", dom);
    htmlparser2.DomUtils.removeElement(items[0]);
    
    // Serialize back to HTML
    const html = htmlparser2.DomUtils.getOuterHTML(dom);
    // "<ul><li class=\"orange\">Orange</li></ul>"
  3. Use WritableStream for streaming input

    master

    To process streaming input, use the WritableStream class from htmlparser2/WritableStream. This allows you to pipe Node.js readable streams directly into the parser.

    import { WritableStream } from "htmlparser2/WritableStream";
    import fs from "fs";
    
    const parserStream = new WritableStream({
        ontext(text) {
            console.log("Streaming:", text);
        },
    });
    
    const htmlStream = fs.createReadStream("./my-file.html");
    htmlStream.pipe(parserStream).on("finish", () => console.log("done"));
  4. Configure Parser options

    master

    When initializing a Parser or using parseDocument, you can provide an options object to control parsing behavior:

    OptionTypeDefaultDescription
    xmlModebooleanfalseTreat the document as XML. Affects entity decoding, self-closing tags, CDATA handling, etc. Set to true for XML, RSS, Atom, and RDF.
    decodeEntitiesbooleantrueDecode HTML entities (e.g. &amp; -> &).
    lowerCaseTagsboolean!xmlModeLowercase tag names.
    lowerCaseAttributeNamesboolean!xmlModeLowercase attribute names.
    recognizeSelfClosingbooleanxmlModeRecognize self-closing tags (e.g. <br/>). Always enabled in xmlMode.
    recognizeCDATAbooleanxmlModeRecognize CDATA sections as text. Always enabled in xmlMode.
  5. Parse RSS, RDF, and Atom feeds

    master

    Use the parseFeed method to quickly parse web feeds. It returns an object containing type, title, link, description, updated, author, and items (an array of entries), or null if the format is unrecognized.

    Note: xmlMode: true is enabled by default for parseFeed. If providing custom options, ensure you include xmlMode: true.

    const feed = htmlparser2.parseFeed(content);
  6. Use the Parser callback interface

    master

    The Parser class provides a low-allocation callback interface. You can instantiate a new Parser by passing a handler object containing optional event callbacks. This is useful for processing documents with minimal memory overhead.

    import * as htmlparser2 from "htmlparser2";
    
    const parser = new htmlparser2.Parser({
        onopentag(name, attributes) {
            if (name === "script" && attributes.type === "text/javascript") {
                console.log("JS! Hooray!");
            }
        },
        ontext(text) {
            console.log("-->", text);
        },
        onclosetag(tagname) {
            if (tagname === "script") {
                console.log("That's it?!");
            }
        },
    });
    
    parser.write("Xyz <script type='text/javascript'>const foo = '<<bar>>';</script>");
    parser.end();
  7. Get a DOM tree with parseDocument

    master

    The parseDocument helper parses a string and returns a DOM tree (a Document node from domhandler). It accepts an optional second argument for both parser and domhandler options (such as withStartIndices or withEndIndices).

    import * as htmlparser2 from "htmlparser2";
    
    const dom = htmlparser2.parseDocument(
        `<ul id="fruits">
            <li class="apple">Apple</li>
            <li class="orange">Orange</li>
        </ul>`,
        {
            // Parser options
            xmlMode: true,
    
            // domhandler options
            withStartIndices: true,
            withEndIndices: true,
        }
    );
  8. Configure parser and handler with Options

    master
    The Options type is a combination of ParserOptions and DomHandlerOptions. This allows you to configure both the low-level tokenizer behavior and the high-level DOM construction behavior simultaneously.
  9. Compare htmlparser2 performance

    master

    For performance comparisons against other HTML parsers based on real-world website benchmarks, refer to the htmlparser-benchmark project. According to recent benchmarks on GitHub Actions, htmlparser2 is one of the fastest options available, outperforming node-html-parser, html5parser, and parse5.

    htmlparser2        : 2.17215 ms/file ± 3.81587
    node-html-parser   : 2.35983 ms/file ± 1.54487
    html5parser        : 2.43468 ms/file ± 2.81501
    neutron-html5parser: 2.61356 ms/file ± 1.70324
    htmlparser2-dom    : 3.09034 ms/file ± 4.77033
    html-dom-parser    : 3.56804 ms/file ± 5.15621
    libxmljs           : 4.07490 ms/file ± 2.99869
    htmljs-parser      : 6.15812 ms/file ± 7.52497
    parse5             : 9.70406 ms/file ± 6.74872
    htmlparser         : 15.0596 ms/file ± 89.0826
    html-parser        : 28.6282 ms/file ± 22.6652
    saxes              : 45.7921 ms/file ± 128.691
    html5              : 120.844 ms/file ± 153.944
  10. Reference Parser events

    master

    The following callbacks can be implemented in the handler object passed to new Parser(handler):

    EventDescription
    onopentag(name, attribs, isImplied)Opening tag. attribs is an object mapping attribute names to values. isImplied is true when the tag was opened implicitly (HTML mode only).
    onopentagname(name)Emitted for the tag name as soon as it is available (before attributes are parsed).
    onattribute(name, value, quote)Attribute. quote is " / ' / null (unquoted) / undefined (no value, e.g. disabled).
    onclosetag(name, isImplied)Closing tag. isImplied is true when the tag was closed implicitly (HTML mode only).
    ontext(data)Text content. May fire multiple times for a single text node.
    oncomment(data)Comment (content between <!-- and -->).
    oncdatastart()Opening of a CDATA section (<![CDATA[).
    oncdataend()End of a CDATA section (]]>).
    onprocessinginstruction(name, data)Processing instruction (e.g. <?xml ...?>).
    oncommentend()Fires after a comment has ended.
    onparserinit(parser)Fires when the parser is initialized or reset.
    onreset()Fires when parser.reset() is called.
    onend()Fires when parsing is complete.
    onerror(error)Fires on error.