Nokogiri Ruby Library

repository·main·Indexed 27 days ago

https://github.com/sparklemotion/nokogiri

A Ruby library for reading, writing, modifying, and querying XML and HTML documents using high-performance native parsers like libxml2 and libgumbo. It features DOM, SAX, and Push parsers, document search via XPath 1.0 and CSS3 selectors, XSD validation, XSLT transformation, and a Builder DSL for document generation. It includes a CLI for parsing files or URIs and supports Ruby >= 3.2 and JRuby >= 10.0.

Tokens
9.8K
Snippets
21
Records
103
Agent score
92%

What's inside Nokogiri

  1. Nokogiri Features Overview

    main

    Nokogiri provides a wide range of tools for working with XML and HTML:

    • DOM Parser: For XML, HTML4, and HTML5.
    • SAX Parser: For XML and HTML4.
    • Push Parser: For XML and HTML4.
    • Document Search: Via XPath 1.0 and CSS3 selectors (with jQuery-like extensions).
    • Validation: XSD Schema validation.
    • Transformation: XSLT transformation.
    • Creation: A "Builder" DSL for generating XML and HTML documents.
  2. Configure the Nokogiri CLI with a configuration file

    main
    The bin/nokogiri command-line tool can be configured by creating a ~/.nokogirirc file. This allows you to define helper methods and customize the environment. For example, you can configure the CLI to use Pry instead of IRB by adding the necessary configuration lines to this file.
  3. Migrate from Nokogiri::HTML to Nokogiri::HTML4

    main

    In version 1.12.0, Nokogiri::HTML was renamed to Nokogiri::HTML4 to distinguish it from the new HTML5 support. While Nokogiri::HTML remains as an alias for backwards compatibility, you should transition to using Nokogiri::HTML4 to prepare for future deprecations.

    Important: If your code relies on checking the class name of an object via Object#class (e.g., checking for a string match), your code may break because objects now report a class of Nokogiri::HTML4::Foo instead of Nokogiri::HTML::Foo.

    Best Practice: Instead of checking class names as strings, use Class#===, Object#is_a?, or Object#instance_of? to ensure compatibility.

  4. Install Nokogiri on ARM64 Linux (AWS Graviton)

    main

    Nokogiri v1.14.0 provides official native gem support for the aarch64-linux platform (ARM64 Linux). This allows for faster and more reliable installations on AWS Graviton and other ARM64 Linux systems.

    Requirement:

    • glibc >= 2.29 is required for aarch64-linux systems.
  5. Install Nokogiri using Native Gems on Linux and OSX

    main

    Starting from version 1.11.0, Nokogiri provides pre-compiled "native gems" for specific architectures. This avoids the need to compile C extensions and packaged libraries locally, resulting in much faster and more reliable installations.

    Supported platforms for native gems:

    • Linux: x86-linux and x86_64-linux (including musl platforms like Alpine)
    • OSX/Darwin: x86_64-darwin and arm64-darwin
    • Windows: Native gems have been supported since 2009.
  6. Use keyword arguments in Nokogiri methods

    main

    Beginning in version 1.17.0 (December 2024), many Nokogiri methods have been modernized to support optional keyword arguments. This allows you to pass specific optional parameters without having to provide values for all preceding positional arguments.

    For example, when using XML::Document.parse, you can now pass options directly without providing nil for url and encoding.

    require 'nokogiri'
    include Nokogiri
    
    xml_s = '<root />'
    options = XML::ParseOptions::STRICT
    
    # Modern way: pass only the keyword argument you need
    XML::Document.parse(xml_s, options: options)
  7. Install Nokogiri

    main

    Nokogiri can be installed via gem install or bundle install. On supported platforms, Nokogiri installs 'native gems' which contain pre-compiled libraries, making installation much faster and more reliable by avoiding the need to compile C extensions or system dependencies.

    Requirements

    • Ruby >= 3.2
    • JRuby >= 10.0
    • If compiling against a system version of libxml2: libxml2 >= 2.9.2 (recommended >= 2.12.0)

    Supported Platforms for Native Gems

    • Linux: x86_64-linux-gnu, aarch64-linux-gnu, arm-linux-gnu (requires glibc >= 2.29), x86_64-linux-musl, aarch64-linux-musl, and arm-linux-musl.
    • Darwin/MacOS: x86_64-darwin and arm64-darwin.
    • Windows: x64-mingw-ucrt.
    • Java: Any platform running JRuby 10.0 or higher.

    You can check your local platform using ruby -e 'puts Gem::Platform.local.to_s'.

    $ gem install nokogiri
  8. Verify Nokogiri dependencies and platform release

    main
    To identify which libraries and platform release you are using, run the nokogiri -v command after installation. For versions >= 1.11.0.rc4, this command emits the complete set of libraries currently in use. This is useful for verifying whether you are using a source-based distribution or a pre-compiled native platform release.
    nokogiri -v
  9. Parse XML strings into Documents or Fragments

    main

    Nokogiri provides two primary ways to parse XML strings or IO objects into a tree of objects:

    1. Nokogiri::XML::parse (shorthand for Nokogiri::XML::Document.parse): Parses a string into a Nokogiri::XML::Document. A document typically has a single root Nokogiri::XML::Element or may be childless. If the input contains multiple top-level tags, only the first one is captured as the root.

    2. Nokogiri::XML::DocumentFragment.parse: Parses a string into a Nokogiri::XML::DocumentFragment. Unlike a document, a fragment can contain multiple immediate child objects of various types (e.g., multiple top-level elements).

  10. Note on HTML4 parser recovery behavior changes

    main
    In Nokogiri v1.13.5 (via libxml2 v2.9.14), the HTML parser's recovery behavior for broken markup changed. Specifically, the XML CDATA escape sequence <![CDATA[ and incorrectly-opened comments may now result in HTML text nodes starting with &lt;! instead of being skipped. This change is a result of a fix for quadratic behavior in the parser. Developers using downstream sanitizers should verify if this change affects their logic.