n8n-nodes-puppeteer

repository·main·Indexed 20 days ago

https://github.com/drudge/n8n-nodes-puppeteer

An n8n node for browser automation using Puppeteer version 1.5.0. It enables users to execute custom scripts, scrape web content, capture screenshots, and generate PDFs by interacting with Chrome/Chromium or Firefox via the DevTools Protocol (CDP) or WebDriver BiDi. The node supports remote browser connections via WebSockets, stealth mode, human typing simulation, and AI agent integration through dynamic script execution.

Tokens
3.7K
Snippets
10
Records
20
Agent score
20%

What's inside n8n-nodes-puppeteer

  1. Use Custom Scripts for Advanced Automation

    main

    The Custom Script operation allows full Puppeteer API access in a sandboxed environment.

    Available Global Objects:

    • $page: Current page instance.
    • $browser: Browser instance.
    • $puppeteer: Puppeteer library.
    • $input.query: Input query from AI agents (when used as a tool).

    AI Agent Integration:

    1. AI-Generated Scripts: Use $fromAI() to let an agent write the code dynamically.
    2. Reusable Scripts: Use $input.query to accept dynamic parameters from an agent.
    // AI agent generates script dynamically
    scriptCode: $fromAI('code', 'Generate a Puppeteer script to extract product prices from the homepage')
    
    // Reusable script using AI input
    const url = $input.query || 'https://example.com';
    await $page.goto(url);
    // ... logic ...
    
    // AI agent provides input via query parameter
    query: $fromAI('url', 'The website URL to scrape')
  2. Available Puppeteer Node Operations

    main

    The Puppeteer node provides several core capabilities for browser automation within n8n workflows:

    • Run Custom Script: Execute arbitrary JavaScript within the browser context.
    • Get Page Content: Extract the HTML or text content of the current page.
    • Get Screenshot: Capture a visual image of the current page.
  3. Manual Installation of n8n-nodes-puppeteer

    main

    To install the node manually in a standard n8n environment (non-Docker):

    1. Navigate to your n8n root directory.
    2. Run npm install n8n-nodes-puppeteer.

    Note on Chromium: By default, Puppeteer downloads a compatible version of Chromium. For production, it is recommended to install system Chrome/Chromium and set the PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true environment variable to reduce installation size and ensure dependency compatibility.

    # Navigate to your n8n root directory
    cd /path/to/n8n
    
    # Install the package
    npm install n8n-nodes-puppeteer
  4. Install n8n-nodes-puppeteer via Docker (Recommended for Production)

    main

    The project provides a pre-configured Docker setup in the docker/ directory that includes all necessary Puppeteer dependencies and Chrome configurations. This is the recommended method for production environments.

    To use this setup:

    1. Clone the repository.
    2. Build the image using npm run docker:build.
    3. Run the container using npm run docker:run (for persistent data) or npm run docker:run:fresh (for a clean start without persistent data).
    # Build the Docker image
    npm run docker:build
    
    # Run n8n with persistent data volume
    npm run docker:run
    
    # Run n8n without persistent data (clean start)
    npm run docker:run:fresh
  5. Install n8n-nodes-puppeteer via Community Nodes

    main

    For n8n version 0.187 and later, you can install the Puppeteer node directly through the n8n UI:

    1. Navigate to Settings > Community Nodes.
    2. Click Install.
    3. Enter n8n-nodes-puppeteer in the Enter npm package name field.
    4. Agree to the risks of using community nodes and select Install.
  6. Connect to a Remote Browser via WebSocket

    main

    If you are using a cloud environment or want to avoid managing Chrome dependencies locally, you can connect to an external Chrome or Firefox instance using a WebSocket endpoint.

    Per-Node Configuration

    In a Puppeteer node, go to Options > Add Option:

    • Browser WebSocket Endpoint: Enter your WebSocket URL (e.g., ws://browserless:3000?token=6R0W53R135510).
    • Protocol:
      • Select CDP (Chrome DevTools Protocol) for Chrome/Chromium.
      • Select WebDriver BiDi for Firefox.

    Global Configuration via Environment Variables

    To apply the connection to all nodes, set the following environment variables:

    For Chrome/Chromium:

    • PUPPETEER_BROWSER_WS_ENDPOINT: The WebSocket URL.
    • PUPPETEER_PROTOCOL: Set to cdp.

    For Firefox:

    • PUPPETEER_BROWSER_WS_ENDPOINT: The WebSocket URL.
    • PUPPETEER_PROTOCOL: Set to webDriverBiDi.
    # Example: Running n8n with a remote Chrome connection via Docker
    docker run -it -p 5678:5678 \
      -e PUPPETEER_BROWSER_WS_ENDPOINT=ws://browserless:3000 \
      -e PUPPETEER_PROTOCOL=cdp \
      n8n-puppeteer
  7. How to use Firefox with Puppeteer

    main

    The node supports Firefox using the webDriverBiDi protocol. To use Firefox, you must have a Firefox instance running with WebDriver BiDi enabled. You can configure the connection in two ways:

    1. Per-node: In the node options, set the Browser WebSocket Endpoint and set the Protocol to webDriverBiDi.
    2. Globally: Set the environment variables PUPPETEER_BROWSER_WS_ENDPOINT and PUPPETEER_PROTOCOL=webDriverBiDi.
  8. Configure Puppeteer via Environment Variables

    main

    You can configure Puppeteer globally using environment variables. This avoids the need to configure every individual node manually. Note that settings configured at the node level will always override these global environment variables.

    docker run -it -p 5678:5678 \
      -e PUPPETEER_BROWSER_WS_ENDPOINT=ws://browserless:3000 \
      -e PUPPETEER_PROTOCOL=cdp \
      n8n-puppeteer
  9. Configure Global Browser Connection and Protocol

    main

    You can control how Puppeteer connects to a browser instance using these options and environment variables:

    • Browser WebSocket Endpoint: The WebSocket URL to connect to. If set, Puppeteer skips launching a local browser.
      • Env vars: PUPPETEER_BROWSER_WS_ENDPOINT or PUPPETEER_WS_ENDPOINT.
    • Protocol: The communication protocol to use.
      • CDP (Chrome DevTools Protocol): Default for Chrome/Chromium.
      • WebDriver BiDi: For Firefox and cross-browser automation.
      • Env var: PUPPETEER_PROTOCOL.
  10. Troubleshoot Missing Shared Library Errors

    main

    If you encounter errors regarding missing shared libraries (e.g., libgobject-2.0.so.0 or libnss3.so), it means the system lacks the necessary dependencies for Chrome/Chromium.

    To resolve this:

    1. Install the missing dependencies on your host/container system.
    2. Switch to a remote browser by using the Browser WebSocket Endpoint option or the PUPPETEER_BROWSER_WS_ENDPOINT environment variable. This offloads the browser execution to a separate environment that already has the dependencies installed.
  11. Configure Remote Browser with Docker Compose

    main

    You can orchestrate n8n and a remote browser (like browserless/chrome) using Docker Compose. This ensures the n8n service is automatically configured to use the browser service via environment variables.

    version: '3.8'
    services:
      n8n:
        image: n8n-puppeteer
        environment:
          - PUPPETEER_BROWSER_WS_ENDPOINT=ws://browserless:3000
          - PUPPETEER_PROTOCOL=cdp
        ports:
          - "5678:5678"
    
      browserless:
        image: browserless/chrome
        ports:
          - "3000:3000"
  12. Manage Cookies between Nodes

    main

    To maintain sessions across different n8n nodes, extract cookies in one node and apply them in another.

    // Node 1: Login and capture cookies
    await $page.goto("https://www.example.com/login");
    await $page.type("#login-username", "user");
    await $page.type("#login-password", "pass");
    await $page.click("#login-button");
    const cookies = await $page.cookies();
    return [{ cookies }];
    
    // Node 2: Restore cookies and access protected page
    const { cookies } = $input.first().json;
    await $page.setCookie(...cookies);
    await $page.goto("https://example.com/protected-page");