Ulixee Hero Documentation

Double Agent is a suite of tools designed to allow a scraper engine to test its detectability by attempting to blend into common web traffic. It operates in two main layers:

1. /collect: Uses scripts and plugins to collect browser profiles by generating a series of pages to test browser behavior.
2. /analyze: Uses scripts and plugins to analyze collected profiles against verified profiles. It compares scraper results to legitimate profiles to find discrepancies, providing a "Looks Human"™ score that indicates the likelihood of being flagged as a bot.

To facilitate the collection phase, the /collect-controller provides a server that generates step-by-step assignments for a scraper to execute all required tests.

A DetachedElement is used for parsing and traversing local HTML fragments without the overhead of a full browser engine. It is a lightweight, W3C-compliant alternative to libraries like Cheerio or JSDOM.

Use DetachedElement when you need to manipulate or query HTML strings or existing DOM fragments in a high-performance environment where a full browser instance is unnecessary.

In the AwaitedDOM API, a Node is a base interface from which various DOM objects inherit (such as Document, Element, Attr, Text, etc.). This allows different types of DOM objects to be treated similarly through a shared set of methods and properties.

Note on Asynchronicity: Many properties and methods on an AwaitedDOM Node return Promise objects. This is a key distinction from the standard synchronous DOM API.

Note on Unimplemented Methods: Some standard DOM methods are currently unimplemented in AwaitedDOM, including:

• appendChild()
• cloneNode()
• insertBefore()
• removeChild()
• replaceChild()
• addEventListener()
• dispatchEvent()
• removeEventListener()

A BrowserContext represents a Chrome Window. In the context of Unblocked Agent, it defaults to an Incognito state.

Key characteristics:

• Incognito by default: Provides isolation from other sessions.
• Proxy Association: An Incognito window can have a dedicated Man-in-the-Middle (MITM) proxy server, which allows for faster and easier association of HTTP requests with specific Page instances within that context.

In AwaitedDOM, a SuperNode is an abstraction that implements the properties and methods of the standard DOM Node interface. It serves as a base interface for various DOM API objects, allowing different types of nodes (such as Document, Element, Text, or Comment) to be treated uniformly.

Key characteristics of SuperNode implementations:

• Inheritance: All SuperNode types inherit from the Node interface, meaning they share a common set of methods and properties.
• Polymorphism: You can interact with diverse DOM objects (e.g., HTMLDivElement, SVGElement, DocumentFragment) using the same Node methods.
• Safety: Methods may return null if a property is not relevant to that specific node type, or they may throw exceptions (for example, if you attempt to add children to a node type that cannot have children).

The Range interface represents a fragment of a document that can contain nodes and parts of text nodes. It allows you to select, manipulate, and query specific portions of the DOM.

You can create a range using:

• Document.createRange()
• The Range() constructor
• Selection.getRangeAt()
• Document.caretRangeFromPoint()