Page Agent

repository·main·Indexed 12 days ago

https://github.com/alibaba/page-agent

An AI-powered GUI agent for web applications that runs directly in-page via JavaScript. It enables developers to integrate AI capabilities like copilots, smart form filling, and accessibility features without backend rewrites or headless browsers. Version 1.12.2 supports custom LLMs and includes an MCP server (@page-agent/mcp) for controlling browsers via clients like Claude Desktop, Cursor, or Copilot.

Tokens
23K
Snippets
70
Records
94
Agent score
98%

What's inside Page Agent

  1. Understand the Page Agent Privacy Model

    main

    Page Agent follows a client-side only architecture with a "Bring Your Own Key" (BYOK) model.

    Key privacy characteristics:

    • No Backend: The software does not include a built-in backend service and does not collect or transmit user data on its own.
    • Data Control: All data transmission occurs exclusively between your browser and the LLM provider you configure. You have full control over which provider receives your data.
    • Local Execution: For the browser extension (Page Agent Ext), DOM analysis and automation actions are performed locally in your browser.
    • Local Storage: Configuration settings (API endpoint, API key, model selection) are stored locally in your browser using chrome.storage.local (or equivalent) and are not synced to any external server.
  2. Data Transmission in Page Agent Ext

    main

    When you initiate an automation task using the Page Agent browser extension, the following data is transmitted to the LLM API endpoint configured in your settings:

    1. Task Instructions: Your natural language commands.
    2. Simplified Page Structure: Cleaned HTML of the pages under the extension's control.

    Security Note on HTML Cleaning: The HTML cleaning process simplifies the structure for AI readability but does not guarantee the removal of sensitive information (such as visible text, form values, or personal data). Always be mindful of the content on the page before initiating a task.

  3. Understand the Page Agent monorepo structure

    main

    Page Agent is a monorepo using npm workspaces, ts references, and vite alias. During development, library package.json exports point to src/*.ts, while published versions point to dist/*.js.

    Published Packages

    • page-agent (packages/page-agent/): The main entry point featuring a built-in UI Panel.
    • @page-agent/mcp (packages/mcp/): An MCP server for controlling the browser via the Page Agent extension.
    • @page-agent/core (packages/core/): Core agent logic (headless, no UI).
    • @page-agent/llms (packages/llms/): LLM client implementing a reflection-before-action mental model.
    • @page-agent/page-controller (packages/page-controller/): Handles DOM operations and visual feedback independently of the LLM.
    • @page-agent/ui (packages/ui/): Decoupled UI components, panels, and i18n.

    Applications

    • Extension (packages/extension/): Browser extension built with WXT and React.
    • Website (packages/website/): React-based documentation, landing page, and dev playground.
  4. How @page-agent/mcp works

    main

    The @page-agent/mcp server acts as a bridge between an AI agent client (via MCP/stdio) and the Page Agent browser extension (via WebSockets).

    Workflow:

    1. Initialization: The agent client starts the server using npx @page-agent/mcp via stdio.
    2. Server Startup: The server starts an HTTP + WebSocket server on localhost:PORT and opens a local launcher page in your browser.
    3. Extension Connection: The launcher page triggers the Page Agent extension to open a hub tab (hub.html?ws=PORT).
    4. Task Proxying: The hub tab connects to the WebSocket server. When the agent client calls an MCP tool, the server proxies that task to the hub tab, which then controls the browser via the extension's useAgent logic.
  5. Core Concepts of Page Agent

    main

    Page Agent is a pure JavaScript GUI agent designed for client-side web enhancement. Unlike traditional automation tools, it is built to run directly within the user's page.

    Key Characteristics

    • No Backend/Plugin Required: Operates entirely via in-page JavaScript without needing a dedicated backend, Python environment, or browser extensions (though an optional Chrome extension is available for cross-page tasks).
    • Text-based DOM Manipulation: It interacts with the page using text-based DOM operations rather than screenshots, which avoids the need for multimodal models or special permissions.
    • Bring Your Own LLM: It supports most mainstream LLMs, including locally deployed models.

    Primary Use Cases

    • SaaS AI Copilot: Add AI capabilities to your product with minimal code.
    • Smart Form Filling: Convert complex multi-step processes into single natural language commands.
    • Accessibility Enhancement: Enable voice commands or screen readers to interact with any webpage using natural language.
    • Cross-page Agent: Use the optional Chrome extension to allow the agent to work across multiple tabs.
    • MCP Integration: Add browser control capabilities to existing agents via the Model Context Protocol (MCP).
  6. Use the Free Testing LLM API for Evaluation

    main

    A free testing LLM API is provided for technical evaluation and R&D purposes. This API is used in the project's live demo and the browser extension's default configuration.

    Critical Restrictions & Warnings:

    • Non-Production Use Only: Do not use this API in any production environment. It is provided "AS IS" and may be rate-limited or discontinued without notice.
    • No Sensitive Data: You are strictly prohibited from inputting Personal Identifiable Information (PII), confidential business data, or financial/medical records.
    • Data Localization Warning: This API processes data via servers located in Mainland China. If you are in a region with strict data localization laws (e.g., EU/EEA), do not use this API.
    • Permitted Use: Use is limited to technical evaluation of the software. Automated scraping at scale or integration into other products is prohibited.

    Recommended Alternative for Real Usage: For secure, continuous, and production-ready usage, use BYOK mode with your own commercial LLM API keys or connect to local, offline models (e.g., Ollama).

  7. Report a security vulnerability in Page Agent

    main

    If you discover a security vulnerability, do not report it through public GitHub issues, discussions, or pull requests.

    Use GitHub's private vulnerability reporting flow:

    1. Navigate to https://github.com/alibaba/page-agent/security/policy.
    2. Click Report a vulnerability.

    If private reporting is unavailable, you may open a minimal public issue to request a private contact channel, but do not include exploit details in the public issue.

  8. Quick Start: One-line integration via CDN

    main

    The fastest way to test Page Agent is by including the demo script via jsDelivr. This version uses a free testing LLM API provided by the maintainers.

    Note: This is for technical evaluation only. By using it, you agree to the terms and privacy policy.

    To prevent the agent from initializing automatically, append ?autoInit=false to the script URL. This allows you to manually instantiate new window.PageAgent(...) with your own LLM configuration.

    <script
        src="https://cdn.jsdelivr.net/npm/page-agent@1.12.2/dist/iife/page-agent.demo.js"
        crossorigin="anonymous"
    ></script>
  9. Develop and build the Page Agent extension

    main

    To work on the browser extension located in packages/extension/, use the following commands:

    npm run dev:ext    # Start extension development mode
    npm run build:ext  # Build the extension

    For details on how to integrate with the extension API, refer to packages/extension/docs/extension_api.md.

    npm run dev:ext
    npm run build:ext
  10. Set up the Page Agent development environment

    main

    To develop locally on Page Agent, ensure you meet the following prerequisites and follow the setup steps.

    Prerequisites

    • OS: macOS, Linux, or WSL
    • Node.js: ^22.13 or >=24
    • npm: >= 11
    • Editor: Must support ts, eslint, and prettier.

    Setup Steps

    Run the following commands in the repository root to install dependencies and start the development environment:

    npm i            # Install dependencies (use `npm ci` to preserve the lockfile)
    npm start        # Start the website development server
    npm run build    # Build the entire monorepo
    npm i
    npm start
    npm run build