puppeteer-real-browser

repository·main·Indexed 23 days ago

https://github.com/zfc-digital/puppeteer-real-browser

A package designed to bypass bot-detecting captchas like Cloudflare by launching Chrome in its most natural state. It uses rebrowser and ghost-cursor to simulate realistic user interactions, such as human-like mouse movements via page.realClick(). Key features include automatic Cloudflare Turnstile solving, support for puppeteer-extra plugins, proxy authentication, and Xvfb integration for Linux environments.

Tokens
3.1K
Snippets
6
Records
14
Agent score
81%

What's inside puppeteer-real-browser

  1. Run with Docker

    main

    The repository includes a Dockerfile tested on Ubuntu server operating systems. To run the project in Docker:

    1. Clone the repository.
    2. Build the image.
    3. Run the container.
    git clone https://github.com/zfcsoftware/puppeteer-real-browser
    cd puppeteer-real-browser
    docker build -t puppeteer-real-browser-project .
    docker run puppeteer-real-browser-project
  2. Configure connect options

    main

    The connect function accepts an options object to customize the browser behavior. Key options include:

    • headless: (Boolean/String) Default is false. While true, new, or shell are supported, false is the most stable for avoiding detection.
    • args: (Array of strings) Additional flags to pass to Chromium (e.g., --start-maximized).
    • customConfig: (Object) Direct initialization arguments for chrome-launcher. Use this to set userDataDir or chromePath.
    • turnstile: (Boolean) If true, Cloudflare Turnstile captchas are automatically clicked.
    • connectOption: (Object) Variables passed to puppeteer.connect (e.g., setting defaultViewport).
    • disableXvfb: (Boolean) In Linux, set to true if you want to see the browser window (disables the virtual screen).
    • ignoreAllFlags: (Boolean) If true, overrides all initialization arguments, including the 'let's get started' page.
    • proxy: (Object) Configuration for proxy usage (commented out in default example).
    const { connect } = require("puppeteer-real-browser");
    
    async function test() {
      const { browser, page } = await connect({
        headless: false,
        args: [],
        customConfig: {},
        turnstile: true,
        connectOption: {},
        disableXvfb: false,
        ignoreAllFlags: false
      });
      await page.goto("<url>");
    }
    
    test();
  3. Troubleshoot viewport and window object issues

    main

    page.setViewport is not working

    If page.setViewport does not behave as expected, use the connectOption to set the defaultViewport. Setting it to null will allow the page to take up the full width of the browser window.

    Cannot access functions in the Window object

    This is likely caused by the runtime being closed by the rebrowser. To resolve this, you can inject JavaScript into the page source using puppeteer-intercept-and-modify-requests or use a Chrome plugin to access the required values.

  4. Install and use Puppeteer-extra plugins

    main

    You can use puppeteer-extra plugins by passing them into the plugins array within the connect options. Note that some plugins (like puppeteer-extra-plugin-anonymize-ua) might increase detection risk.

    Example installation and usage:

    npm i puppeteer-extra-plugin-click-and-wait
    const { connect } = require("puppeteer-real-browser");
    const clickAndWaitPlugin = require("puppeteer-extra-plugin-click-and-wait");
    
    async function test() {
      const { page, browser } = await connect({
        args: ["--start-maximized"],
        turnstile: true,
        headless: false,
        customConfig: {},
        connectOption: {
          defaultViewport: null,
        },
        plugins: [clickAndWaitPlugin()],
      });
      await page.goto("https://google.com", { waitUntil: "domcontentloaded" });
      await page.clickAndWaitForNavigation("body");
      await browser.close();
    }
    
    test();
    const test = require("node:test");
    const assert = require("node:assert");
    const { connect } = require("puppeteer-real-browser");
    
    test("Puppeteer Extra Plugin", async () => {
      const { page, browser } = await connect({
        args: ["--start-maximized"],
        turnstile: true,
        headless: false,
        // disableXvfb: true,
        customConfig: {},
        connectOption: {
          defaultViewport: null,
        },
        plugins: [require("puppeteer-extra-plugin-click-and-wait")()],
      });
      await page.goto("https://google.com", { waitUntil: "domcontentloaded" });
      await page.clickAndWaitForNavigation("body");
      await browser.close();
    });
  5. Use the connect function

    main

    The primary entry point is the connect function. It returns an object containing the page and browser instances. You can use it with CommonJS or ES Modules.

    CommonJS:

    const { connect } = require("puppeteer-real-browser");
    const { page, browser } = await connect();

    ES Modules:

    import { connect } from "puppeteer-real-browser";
    const { page, browser } = await connect();
    const { connect } = require("puppeteer-real-browser");
    
    const start = async () => {
      const { page, browser } = await connect();
    };
  6. Use pageController to manage page behavior and evasion

    main

    The pageController function is an asynchronous utility used to initialize a Puppeteer page with advanced evasion features, proxy authentication, plugin support, and human-like cursor movements.

    When called, it performs the following actions:

    • Proxy Authentication: If proxy.username and proxy.password are provided, it automatically authenticates the page.
    • Plugin Execution: Iterates through the plugins array and calls plugin.onPageCreated(page) for each.
    • Evasion Injection: Injects scripts into the page context to fix MouseEvent properties (screenX and screenY) to prevent detection.
    • Human-like Interaction: Attaches a realCursor (via ghost-cursor) to the page object, providing a page.realClick method for non-bot-like clicking.
    • Turnstile Solving: If turnstile is truthy, it starts a background loop that periodically checks for and attempts to solve Cloudflare Turnstile challenges.
    • Process Cleanup: If killProcess is set to true, it will attempt to kill the xvfbsession, chrome instance, and the provided pid when the browser disconnects.
  7. Handle Cloudflare Turnstile challenges with checkTurnstile

    main

    The checkTurnstile function is used to detect and interact with Cloudflare Turnstile captcha elements on a page. It attempts to find the Turnstile response field or identifies the Turnstile widget by scanning for specific div dimensions and styles. If found, it performs mouse clicks on the identified coordinates to attempt to solve the challenge.

    Behavior:

    • It waits up to 5000ms for the Turnstile element to appear.
    • If the [name="cf-turnstile-response"] element is found, it clicks within its parent container.
    • If the response element is not found, it falls back to scanning the DOM for div elements that match the typical Turnstile widget dimensions (width between 290px and 310px) and clicks them.
    • Returns true if it successfully identifies and interacts with a challenge, or false if no challenge is found or an error occurs.
  8. Use pageController to initialize a browser page with evasion features

    main

    The pageController function is used to set up a Puppeteer page instance with advanced evasion capabilities, including proxy authentication, plugin support, Turnstile solving, and realistic mouse movements.

    When called, it performs the following setup:

    • Proxy Authentication: If proxy.username and proxy.password are provided, it automatically authenticates the page.
    • Plugin Execution: Iterates through the plugins array and calls plugin.onPageCreated(page) for each.
    • Evasion Scripts: Injects a script via page.evaluateOnNewDocument to spoof MouseEvent properties (screenX and screenY) to prevent detection.
    • Real Cursor: Attaches a ghost-cursor instance to the page object as page.realCursor and provides a convenience method page.realClick for human-like clicking.
    • Turnstile Solving: Starts an asynchronous loop to check for and solve Cloudflare Turnstile challenges if turnstile is truthy.
    • Process Cleanup: If killProcess is set to true, it attempts to clean up the xvfbsession, chrome process, and the main pid when the browser disconnects.
  9. Connect to a real browser using connect()

    main

    The connect() function is the primary entrypoint for puppeteer-real-browser in CommonJS environments. It launches a Chrome instance configured to bypass automation detection and returns a controlled browser and page object.

    Key features include:

    • Anti-detection: Automatically modifies Chrome flags (like adding AutomationControlled to --disable-features) to appear more human-like.
    • Xvfb Support: On Linux, it can automatically start an Xvfb session to provide a virtual display if disableXvfb is not set to true.
    • Plugin Support: Allows passing an array of puppeteer-extra plugins which are automatically applied via puppeteer.use().
    • Proxy Support: Configures the browser to use a proxy via the proxy object.
    • Turnstile Support: Enables Turnstile-specific handling when turnstile: true is passed.

    Note on Linux: If running on Linux without Xvfb, the function will attempt to start it but will log an error suggesting sudo apt-get install xvfb if it is missing.

  10. Use connect() to launch a real browser session

    main
    The connect() function is the primary entry point for puppeteer-real-browser. It initializes a browser instance and returns a ConnectResult object containing both the browser and a specialized page object. The returned page is a PageWithCursor, which extends the standard Puppeteer Page with human-like interaction capabilities like realClick and realCursor.