DOCSEARCHbyAlgolia
Submit Docs

Nokolexbor Documentation

repository·master·Indexed 19 days ago

https://github.com/serpapi/nokolexbor

A high-performance HTML5 parser for Ruby designed as a drop-in replacement for Nokogiri. It leverages the Lexbor engine for faster HTML parsing and CSS selector execution, providing a 1:1 compatible API for parsing and searching nodes via CSS and XPath. Features include a DSL-based HTML builder, a fluent NodeBuilder API for attributes, and a NodeSet for bulk DOM manipulation.

Tokens
6.4K
Snippets
24
Records
29
Agent score
64%

What's inside Nokolexbor

  1. Compare CSS and XPath searching methods

    master

    Nokolexbor offers different searching engines depending on the method used:

    css and at_css (Lexbor-based)

    • High Performance: Significantly faster than libxml2-based methods.
    • Constraint: Only accepts pure CSS selectors. It does not support mixed syntax like div#abc /text().
    • Text Nodes: To select text nodes, use the ::text pseudo-element (e.g., div#abc > ::text).

    xpath and at_xpath (libxml2-based)

    • Standard XPath: Only accepts XPath syntax.
    • Behavior: Works identically to Nokogiri's xpath and at_xpath methods.

    nokogiri_css and nokogiri_at_css (libxml2-based)

    • Requirement: Requires the nokogiri gem to be installed.
    • Mixed Syntax: Supports mixed syntax like div#abc /text().
    • Behavior: Works identically to Nokogiri's CSS methods.

  2. Note on `:nth-of-type(n)` behavior

    master

    Nokolexbor's CSS selector engine follows browser behavior regarding :nth-of-type(n). The index n is not affected by prior filters (like :not).

    For example, if you want to select the 3rd div that does not have class a or b, in Nokogiri you might use div:not(.a):not(.b):nth-of-type(3). In Nokolexbor, you must account for the excluded elements in the index, e.g., div:not(.a):not(.b):nth-of-type(5).

  3. Install Nokolexbor

    master

    Nokolexbor is available as a pre-compiled gem on most common platforms (Linux x86_64/aarch64, macOS x86_64/arm64, and Windows ucrt64).

    To install via Gemfile:

    gem 'nokolexbor'

    Then run bundle install.

    To install directly via CLI:

    gem install nokolexbor

    If you are on an unsupported platform, you must install cmake to compile the C extensions.

    gem install nokolexbor

  4. Quick start with Nokolexbor

    master

    Nokolexbor provides a 1:1 compatible API with Nokogiri for parsing HTML and searching nodes. Use Nokolexbor::HTML() to parse a document and then use .css() or .xpath() to query it.

    require 'nokolexbor'
require 'open-uri'

# Parse HTML document
doc = Nokolexbor::HTML(URI.open('https://github.com/serpapi/nokolexbor'))

# Search for nodes by css
doc.css('#readme h1', 'article h2', 'p[dir=auto]').each do |node|
  puts node.content
end

# Search for text nodes by css
doc.css('#readme p > ::text').each do |text|
  puts text.content
end

# Search for nodes by xpath
doc.xpath('//div[@id="readme"]//h1', '//article//h2').each do |node|
  puts node.content
end

  5. Build HTML documents with Nokolexbor::Builder

    master

    The Nokolexbor::Builder class provides a Domain Specific Language (DSL) for programmatically constructing HTML documents. It supports two primary styles of block usage:

    1. No-arg block (instance_eval style): Tag names are called as bare methods. This is the most concise syntax.
    2. Block-parameter style: Tag names are called on the builder argument passed into the block.

    You can also build content into an existing node using Nokolexbor::Builder.with(existing_node).

    # No-arg block style
Nokolexbor do
  body do
    h1 'Hello world'
    p 'This little p'
    ul do
      li 'Go to market'
      li 'Go to bed'
    end
  end
end

# Block-parameter style
Nokolexbor::Builder.new do |b|
  b.body do
    b.h1 'Hello world'
  end
end

# Building into an existing node
Nokolexbor::Builder.with(existing_node) do
  span 'injected'
end

  6. Troubleshoot glibc compatibility errors

    master

    If you encounter a LoadError related to GLIBC, it means the precompiled native gem is incompatible with your system's glibc version. To resolve this, you must install the gem using the ruby platform instead of the precompiled native version.

    Fix via CLI:

    gem install nokolexbor --platform=ruby

    Fix via Bundler:

    bundle config set force_ruby_platform true

  7. Manage document meta encoding

    master

    You can retrieve or set the character encoding of the document using meta_encoding.

    • Get encoding: meta_encoding returns the charset from the <meta charset=...> tag or the Content-Type meta tag. Returns nil if no encoding is specified.
    • Set encoding: meta_encoding= sets the encoding. If a meta tag doesn't exist, the method will automatically attempt to create one and insert it into the <head> or appropriate location in the document structure.
    # Get current encoding
encoding = doc.meta_encoding

# Set a new encoding
doc.meta_encoding = "UTF-8"

  8. Register namespaces in XPathContext

    master

    The Nokolexbor::XPathContext#register_namespaces method allows you to register multiple XML namespaces at once using a hash. When providing the hash, the keys can include prefixes like xmlns: or xml:; the method automatically strips these prefixes before registering the namespace. This is used to ensure that XPath queries involving prefixed elements can be correctly evaluated against the document's namespaces.

    # Example usage of register_namespaces
context = Nokolexbor::XPathContext.new
context.register_namespaces({
  'xmlns:prefix' => 'http://example.com/schema',
  'xml' => 'http://www.w3.org/XML/1998/namespace'
})

  9. Manipulate collections of nodes with NodeSet

    master

    A NodeSet is a collection of Nokolexbor::Node objects returned by CSS or XPath queries. It includes the Enumerable module, allowing you to iterate over nodes, search for specific elements, and perform bulk operations on all nodes in the set.

    Common Operations

    • Iteration: Use #each to iterate or #to_a to convert to an array.
    • Selection: Use #first(n) to get the first n elements (or just the first if n is nil) and #last to get the final element.
    • Bulk Content Extraction:
      • #content (aliases: #text, #inner_text, #to_str): Returns the concatenated text content of all nodes.
      • #inner_html: Returns the concatenated inner HTML of all nodes.
      • #outer_html (aliases: #to_s, #to_html, #serialize): Returns the concatenated outer HTML of all nodes.
    • Bulk Attribute Manipulation:
      • #attr(key, value = nil, &block) (aliases: #set, #attribute):
        • If key is a Hash, it sets multiple attributes on all nodes.
        • If key is a single value and value is provided, it sets that attribute on all nodes.
        • If value is nil, it retrieves the attribute from the first node in the set.
        • Supports a block for dynamic value assignment.
      • #add_class(name) / #append_class(name): Adds a class to all nodes.
      • #remove_class(name = nil): Removes a specific class from all nodes.
      • #remove_attr(name) (alias: #remove_attribute): Removes a specific attribute from all nodes.
    • Bulk DOM Modification:
      • #remove (alias: #unlink): Removes all nodes in the set from the document.
      • #destroy: Destroys all nodes in the set.
      • #wrap(node_or_tags): Wraps every node in the set with the specified node or tags.
      • #add_class(name) / #append_class(name): Adds a class to all nodes.

    Searching within a NodeSet

    You can perform nested searches starting from the nodes within a NodeSet using:

    • #xpath(*args)
    • #nokogiri_css(*args)
    # Get an attribute from the first node in a NodeSet
value = node_set.attr("href")

# Set attributes on all nodes
node_set.attr("href" => "http://example.com", "class" => "a")

# Set attributes using a block
node_set.attr("href") { |node| "http://example.com" }

# Get concatenated text
text = node_set.text

# Get concatenated HTML
html = node_set.to_html

  10. Handle XPath SyntaxError in Nokolexbor

    master

    When performing XPath queries, Nokolexbor may raise a Nokolexbor::XPath::SyntaxError. This error object provides detailed metadata about the failure, including the location (line and column) and the severity level of the issue. You can use predicate methods to determine if the issue is a warning, a standard error, or a fatal error.

    begin
  # Perform XPath operation that might fail
  document.xpath('//invalid[xpath')
rescue Nokolexbor::XPath::SyntaxError => e
  puts "Location: #{e.line}:#{e.column}"
  puts "Severity: #{e.level}"
  
  if e.fatal?
    puts "A fatal error occurred: #{e.message}"
  elsif e.error?
    puts "A standard error occurred: #{e.message}"
  elsif e.warning?
    puts "A warning occurred: #{e.message}"
  end
end

  11. Use the fluent NodeBuilder API for attributes and classes

    master

    When creating elements within a builder block, Nokolexbor returns a NodeBuilder object that allows for a fluent, chainable API to set classes, IDs, and other attributes.

    • Classes: Call the class name as a method: div.container becomes <div class="container">.
    • IDs: Append an exclamation mark to the method name: div.thing! becomes <div id="thing">.
    • Multiple Classes: Chain methods: div.box.highlight becomes <div class="box highlight">.
    • Attributes: Use the []= operator or pass an options hash to set arbitrary attributes.
    • Content: Pass a string as an argument to the method to set the element's text content.
    # Examples of fluent NodeBuilder usage:
div.container          # <div class="container">
div.box.highlight     # <div class="box highlight">
div.thing!            # <div id="thing">
div.container.hero!   # <div class="container" id="hero">

# Setting attributes via hash
div.container(id: 'main', data_role: 'wrapper')

# Setting content
div.title 'My Heading'

  12. Manage CSS classes on a Node

    master

    Nokolexbor provides convenience methods for managing space-delimited CSS classes on an element's class attribute.

    • classes: Returns an Array of the current CSS classes.
    • add_class(names): Adds one or more classes. It ensures no duplicates are added. Accepts a String or Array of strings.
    • append_class(names): Adds classes regardless of whether they already exist (allows duplicates).
    • remove_class(names): Removes specified classes. If no classes remain, the class attribute is deleted. Accepts a String or Array of strings.
    node = doc.at_css('div')

node.classes          # => ["old-class"]
node.add_class("new") # <div class="old-class new"></div>
node.add_class("new") # <div class="old-class new"></div> (no duplicate)
node.append_class("new") # <div class="old-class new new"></div> (duplicates allowed)
node.remove_class("old-class") # <div class="new"></div>