Playwriter Documentation

repository·main·Indexed 25 days ago

https://github.com/remorses/playwriter

A browser automation tool for AI agents that connects to existing Chrome instances via a Chrome extension and MCP. It allows agents to utilize existing logins, extensions, and cookies while bypassing bot detection. Features include a CLI for executing Playwright commands, stateful session management, accessibility labels (aria-ref) for token-efficient element identification, CDP session debugging, and RTMP streaming of browser tabs.

Tokens
82.1K
Snippets
252
Records
406
Agent score
86%

What's inside Playwriter

  1. Understand the Playwriter Security Model

    main

    Playwriter operates on a local-only architecture. The relay server binds to localhost:19988 and only accepts connections from the Playwriter extension via WebSocket.

    Key security features include:

    • Origin Validation: The extension WebSocket endpoint only accepts the Playwriter extension origin to prevent malicious websites from controlling your browser.
    • Explicit Consent: Only tabs where you have explicitly clicked the extension icon are controlled. There is no background access to other tabs.
    • Visible Automation: Chrome displays an automation banner on all controlled tabs, allowing you to monitor agent activity in real time.
  2. Understand Playwright's Accessibility Snapshot Implementation

    main

    Playwright implements accessibility snapshots via the ariaSnapshot() method. Instead of using Chrome DevTools Protocol (CDP) commands, Playwright uses browser-side JavaScript injection. An injected script traverses the DOM directly using browser APIs to generate the tree.

    Key characteristics:

    • Cross-browser compatibility: Works in Firefox, WebKit, and Chromium.
    • Limitations: It cannot automatically traverse into <iframe> content due to security restrictions and design choices. It detects iframes and adds them to the tree with role: 'iframe', but the children array for an iframe node is always empty.
  3. Understand Page and Context in Playwright

    main

    Automation in Playwright is managed through two primary objects:

    • page: Represents a single browser tab.
    • context: Represents a browser session, containing cookies, storage, and other session-specific data.

    You can create a new page from an existing context using context.newPage().

    // Assuming you have page and context already available
    const page = await context.newPage()
  4. Understand Playwriter's connection mechanism

    main
    Playwriter connects browser tabs to local Playwright automation scripts using the Chrome DevTools Protocol (CDP). It establishes a WebSocket connection to ws://localhost:19988 on your local machine. This connection acts as a local IPC (inter-process communication) channel to pass JSON command messages between your local Playwright scripts and the browser. It does not download or execute remote code; all extension logic is bundled locally.
  5. Compare Playwriter vs Playwright CLI

    main

    Playwriter is designed for agent-driven browser control by running the Playwright API against your existing Chrome instance, whereas the Playwright CLI (npx playwright) spawns a fresh, isolated browser instance.

    Key differences include:

    • Browser State: Playwright CLI starts with a fresh (logged out) browser; Playwriter uses your existing Chrome with all cookies and login states intact.
    • Extensions: Playwright CLI has no extensions; Playwriter uses your existing Chrome extensions.
    • Captchas: Playwright CLI is often blocked by captchas; Playwriter can bypass them by using your existing session.
    • Capabilities: Playwriter provides raw CDP access and native high-FPS video recording, which the Playwright CLI lacks.
  6. Security model of Playwriter

    main

    Playwriter is designed to run locally on your machine with the following security constraints:

    • Local only: The WebSocket server binds to localhost:19988. No data leaves your machine to a remote server.
    • Origin validation: Only the Playwriter extension origin is accepted, preventing malicious websites from spoofing the connection.
    • Explicit consent: The agent only controls tabs where you have manually clicked the extension icon; it has no background access.
    • Visible automation: Chrome displays an automation banner on all controlled tabs.
  7. Security and Privacy features

    main

    Playwriter is designed to run locally on your machine with the following security constraints:

    • Local-only WebSocket: The relay binds to localhost:19988 and only accepts connections from the extension.
    • Origin validation: Only the Playwriter extension origin is accepted, preventing malicious websites from spoofing connections.
    • Explicit consent: Automation only occurs on tabs where you have manually clicked the extension icon; there is no background access.
    • Visible automation: Chrome displays an automation banner on all controlled tabs.
  8. Automate repetitive browser tasks

    main

    Playwriter can automate tasks that require a logged-in session, such as downloading data from specific URLs, bulk filling forms from external data sources, or exporting files from authenticated dashboards.

    To bulk fill forms using an external script, use the -f flag.

    # Download a YouTube playlist
    playwriter -s 1 -e $'state.page = context.pages().find(p => p.url() === "about:blank") ?? await context.newPage()
    await state.page.goto("https://www.youtube.com/playlist?list=PLxxxxxx")
    await waitForPageLoad({ page: state.page, timeout: 5000 })
    const videos = await state.page.$$eval("a#video-title", els => els.map(e => ({ title: e.textContent.trim(), href: e.href })))
    console.log(JSON.stringify(videos, null, 2))
    '
    
    # Bulk fill forms
    playwriter -s 1 -f fill-forms.js
    
    # Export data from authenticated dashboards
    playwriter -s 1 -e $'state.page = context.pages().find(p => p.url() === "about:blank") ?? await context.newPage()
    await state.page.goto("https://analytics.example.com/export")
    const [download] = await Promise.all([
      state.page.waitForEvent("download"),
      state.page.click("button:has-text(\"Export CSV\")")
    ])
    await download.saveAs("/tmp/analytics.csv")
    console.log("Saved to /tmp/analytics.csv")
    '
  9. Trigger browser downloads for large data

    main

    For large datasets that might truncate in console output, use state.page.evaluate to create a Blob and trigger a client-side download link.

    await state.page.evaluate(async url => {
      const resp = await fetch(url)
      const data = await resp.text()
      const blob = new Blob([data], { type: 'application/octet-stream' })
      const a = document.createElement('a')
      a.href = URL.createObjectURL(blob)
      a.download = 'data.json'
      a.click()
    }, 'https://example.com/protected/large-file')
  10. Configure a remote machine to control a host browser

    main

    Once the host machine is running, you can control it from a remote machine using the playwriter CLI. You can configure the connection using environment variables or command-line flags.

    Using Environment Variables

    Set the PLAYWRITER_HOST and PLAYWRITER_TOKEN variables:

    export PLAYWRITER_HOST=https://my-machine-tunnel.traforo.dev
    export PLAYWRITER_TOKEN=MY_SECRET_TOKEN

    Then run commands using the -s (session) flag. Use playwriter session new to generate a new session ID:

    playwriter session new          # outputs: 1
    playwriter -s 1 -e "await page.goto('https://example.com')"

    Using CLI Flags

    Alternatively, pass the connection details directly as flags:

    playwriter --host https://my-machine-tunnel.traforo.dev --token MY_SECRET_TOKEN -s 1 -e "await page.goto('https://example.com')"
    export PLAYWRITER_HOST=https://my-machine-tunnel.traforo.dev
    export PLAYWRITER_TOKEN=MY_SECRET_TOKEN
    
    playwriter session new          # outputs: 1
    playwriter -s 1 -e "await page.goto('https://example.com')"
  11. Add or update @xmorse/playwright-core public APIs

    main

    The @xmorse/playwright-core package uses a generated types.d.ts file. To add or update public APIs, follow this exact sequence to ensure TypeScript consumers can see the changes:

    1. Implement runtime code in playwright/packages/playwright-core/src/client/*.ts (client-side), playwright/packages/playwright-core/src/server/*.ts (server-side), and playwright/packages/protocol/src/channels.d.ts (protocol) if necessary.
    2. Add a doc entry in Markdown (the source of truth) in playwright/docs/src/api/class-*.md. Use * langs: js for JS/TS-only APIs.
    3. Add type overrides in playwright/utils/generate_types/overrides.d.ts for standalone exported types, complex generics, or function overloads.
    4. Regenerate types.d.ts using node playwright/utils/generate_types/index.js.
    5. Rebuild playwright-core using pnpm playwright:build.
    6. Add a changeset for @xmorse/playwright-core.
    7. Verify by running pnpm typecheck in the playwriter/ package.
  12. Control Chrome on a LAN

    main

    You can control Chrome over a Local Area Network (LAN) without using tunnels. Set the PLAYWRITER_HOST environment variable to the IP address of the host machine.

    For MCP (Model Context Protocol) clients, set both PLAYWRITER_HOST and PLAYWRITER_TOKEN in your MCP client's environment configuration.