mcp-server-browser-use

repository·main·Indexed 21 days ago

https://github.com/saik0s/mcp-browser-use

An MCP server (version 0.3.0) that enables AI assistants to control a web browser via the browser-use library. It provides tools for browser automation, deep research, and an experimental skills system to record and replay tasks. The server uses an HTTP transport for long-running tasks and includes a web UI dashboard for monitoring task progress and managing learned skills.

Tokens
26.1K
Snippets
82
Records
104
Agent score
76%

What's inside mcp-server-browser-use

  1. What is a 'Money Request'?

    main

    A money request is the specific API call that returns the data requested by the user. It is the core component of a skill. A machine-generated money request in a skill file includes:

    • endpoint: The URL path.
    • method: The HTTP method (e.g., POST).
    • identifies_by: A way to distinguish the request (e.g., an operationName).
    • response_path: The JSON path to the relevant data in the response body.
    money_request:
      endpoint: "/api/graphql/v1"
      method: POST
      identifies_by: "operationName: searchJobs"
      response_path: "data.searchJobs.edges"
  2. Monitor and query task observability

    main

    The server tracks all tool executions in a SQLite database (~/.config/mcp-server-browser-use/tasks.db) for debugging and monitoring. Tasks progress through a lifecycle (PENDINGRUNNINGCOMPLETED/FAILED/CANCELLED) and granular stages (INITIALIZINGPLANNINGNAVIGATINGEXTRACTINGSYNTHESIZING).

    You can query task status via the CLI or through MCP tools available to AI clients:

    • mcp-server-browser-use tasks: List recent tasks.
    • mcp-server-browser-use task <task_id>: Get full details of a specific task.
    • mcp-server-browser-use health: Check server health (uptime, memory, running tasks).

    MCP Tools for AI Clients:

    • health_check: Server status + list of running tasks.
    • task_list: Recent tasks with optional status filter.
    • task_get: Full details of a specific task.
  3. How skill-based execution works

    main

    Skills allow you to transition from slow AI-driven exploration to fast direct execution:

    1. Discovery: When learn: true is used, the agent explores the site and extracts API patterns/network calls.
    2. Saving: The patterns are saved as a skill using save_skill_as.
    3. Direct Execution: Subsequent calls using skill_name bypass the AI agent and use the extracted API patterns directly (reducing execution time from ~60-120s to ~2-5s).
    4. Fallback: If direct execution fails (e.g., due to authentication changes), the system automatically falls back to the standard AI agent-based execution.
  4. Handle Progress reporting with `Progress()`

    main

    To report progress in a tool, use Progress() as the default value for the progress parameter.

    Important Rules:

    1. Do not use Optional[Progress] = None: This bypasses FastMCP's dependency injection system and may cause tools to fail in background task modes.
    2. Always check for availability: Before calling methods on the progress object, check if it exists (if progress:) to avoid crashes when progress is not available.
    3. Always await progress methods: All progress operations (e.g., set_total, set_message, increment) are asynchronous.

    Recommended Helper Pattern for Graceful Degradation: If building a class that needs to report progress, implement a helper that checks for the existence of the progress object before attempting operations.

    from fastmcp.dependencies import Progress
    from typing import Optional
    
    @server.tool(task=TaskConfig(mode="optional"))
    async def long_running_task(
        topic: str,
        progress: Progress = Progress(),  # ✅ Correct
    ) -> str:
        if progress:
            await progress.set_total(100)
            await progress.set_message("Starting task...")
    
        # Do work...
    
        if progress:
            await progress.increment()
    
        return "Complete"
  5. Understand the mcp-server-browser-use architecture

    main

    The project operates as a FastMCP server that communicates with MCP clients (like Claude Desktop or CLI tools) via HTTP POST requests to /mcp.

    At its core, the server exposes MCP tools such as run_browser_agent and run_deep_research. These tools leverage the browser-use library (Agent + Playwright) to perform web automation. The architecture is composed of several key modules:

    • FastMCP Server: The entry point providing tools and managing connections.
    • Providers: A factory supporting 12 different LLM providers (including OpenAI, Anthropic, Gemini, Groq, and Ollama).
    • Skills: A system for learning and running browser skills using CDP network capture and LLM extraction.
    • Research: A workflow for deep research involving planning, searching, and synthesizing results.
    • Observability: Task tracking and persistence using SQLite.
    ┌─────────────────────────────────────────────────────────────────────────┐
    │                           MCP CLIENTS                                    │
    │              (Claude Desktop, mcp-remote, CLI call)                      │
    └─────────────────────────────────┬───────────────────────────────────────┘
                                      │ HTTP POST /mcp
                                      ▼
    ┌─────────────────────────────────────────────────────────────────────────┐
    │                         FastMCP SERVER                                   │
    │  ┌──────────────────────────────────────────────────────────────────┐   │
    │  │                      MCP TOOLS                                    │  │
    │  • run_browser_agent    • skill_list/get/delete                    │  │
    │  • run_deep_research    • health_check/task_list/task_get          │  │
    │  └──────────────────────────────────────────────────────────────────┘   │
    └────────┬──────────────┬─────────────────┬────────────────┬──────────────┘
             │              │                 │                │
             ▼              ▼                 ▼                ▼
    ┌─────────────┐  ┌─────────────┐  ┌─────────────────┐  ┌─────────────────────┐
    │   CONFIG    │  │  PROVIDERS  │  │     SKILLS      │  │    OBSERVABILITY    │
    │  Pydantic   │  │ 12 LLMs     │  │  Learn+Run      │  │   Task Tracking     │
    └─────────────┘  └─────────────┘  └─────────────────┘  └─────────────────────┘
                                             │
                                             ▼
                                  ┌─────────────────────────┐
                                  │      browser-use        │
                                  │   (Agent + Playwright)  │
                                  └─────────────────────────┘
  6. Best practices for FastMCP dependency injection and progress

    main

    To avoid common runtime errors in FastMCP, follow these patterns:

    Dependency Injection

    • Never use static instances: Do not use ctx: Context = Context(). This creates a static instance that isn't linked to the runtime.
    • Use Sentinels: Always use CurrentContext() as the default value for context injection so the FastMCP runtime can properly inject the request-specific context.

    Progress Reporting

    • Defensive Checks: FastMCP might not inject a Progress object if it's not needed. Always wrap progress operations in a check: if progress: ....
    • Avoid Optional Defaults: Do not use progress: Optional[Progress] = None. This prevents the runtime from injecting the object. Instead, use progress: Progress = Progress() and check for its existence manually.
  7. Skill lifecycle and status field

    main

    Skills in the system include a status field to track their reliability. The status field is a Literal that can be one of:

    • draft: The default status for newly created skills.
    • verified: Set after a skill has successfully executed for the first time.
    • failed: Set after a skill has encountered 3 consecutive failures.

    This status is persisted in the skill's YAML configuration and can be used to filter skills via the skill_list tool.

  8. How dependency injection and progress tracking work in FastMCP

    main

    FastMCP uses a signature-inspection mechanism to handle dependency injection and background task progress.

    Dependency Injection Flow

    1. The MCP client calls a tool.
    2. FastMCP inspects the tool's function signature.
    3. If it finds ctx: Context = CurrentContext(), it injects the current request's Context.
    4. If it finds progress: Progress = Progress(), it injects a real Progress tracker (if in task mode) or a no-op tracker.
    5. The tool executes with these injected objects.

    Progress Tracking Flow

    1. A client requests a background task (using task=True).
    2. FastMCP creates a Progress tracker.
    3. The tool receives the injected Progress instance.
    4. The tool updates the client via progress.set_total(N), progress.set_message("status"), and progress.increment().
    5. Updates are sent to the client via the MCP protocol.
  9. How the Skills feature works

    main

    The Skills feature allows mcp-browser-use to learn browser tasks once and replay them efficiently using API hints rather than DOM scraping.

    Skills are machine-generated by an agent during a 'Learning Mode' session. Instead of reading HTML elements, the agent is instructed to discover the underlying API endpoints (the 'money request') that return the required data. Once discovered, these details are saved as YAML files.

    In 'Execution Mode', the agent uses these saved skills to navigate directly to the correct state and target the specific API endpoint, making execution faster, more reliable, and less prone to UI changes.

    /* 
    Architecture Overview:
    
    LEARNING MODE (learn=True):
    Agent + API Focus Instructions -> Analyzer (LLM finds money req) -> Store (YAML skill files)
    
    EXECUTION MODE (skill_name provided):
    Store (load skill) -> Executor (inject hints) -> Agent + Hints
    */
  10. Integrate progress reporting in internal logic

    main

    To report progress from internal classes (like a ResearchMachine) to the MCP client, pass the Progress object into the class constructor. Use an internal helper method to safely report updates.

    Key patterns:

    • Use Optional["Progress"] for type hints to avoid circular imports.
    • Always check if not self.progress: before attempting to report.
    • Use methods like set_total, set_message, and increment to update the client.
    class ResearchMachine:
        def __init__(self, ..., progress: Optional["Progress"] = None):
            self.progress = progress
    
        async def _report_progress(self, message: Optional[str] = None, increment: bool = False, total: Optional[int] = None) -> None:
            if not self.progress:
                return
            if total is not None:
                await self.progress.set_total(total)
            if message:
                await self.progress.set_message(message)
            if increment:
                await self.progress.increment()
  11. Implement FastMCP tools with progress tracking

    main

    When implementing tools using FastMCP to support background execution and progress reporting, follow these mandatory patterns:

    1. Context Injection: Use CurrentContext() as the default value for the context parameter, not Context().
    2. Progress Injection: Use Progress() as the default value for the progress parameter, not None.
    3. Task Configuration: Use TaskConfig(mode="optional") in the @server.tool decorator to allow the tool to run both synchronously and as a background task.
    4. Progress Safety: Always check if progress: before calling any progress methods (e.g., set_total, set_message, increment), as it may be a no-op if the client didn't request it.
    @server.tool(task=TaskConfig(mode="optional"))
    async def my_tool(
        arg: str,
        ctx: Context = CurrentContext(),
        progress: Progress = Progress(),
    ) -> str:
        if progress:
            await progress.set_message("Working...")
        # ... logic ...
        return "Done"
  12. How client-visible status updates work via Context

    main

    The mcp-server-browser-use project uses the FastMCP Context object to provide real-time visibility into long-running browser automation or research tasks. Instead of waiting for a task to complete (which can take 30s to 10min), the server uses ctx.info() to send milestone messages directly to the MCP client.

    To avoid overwhelming the client with noise, the implementation follows a 'signal, not noise' principle:

    • Browser Automation: Only logs page transitions (URL/Title changes) rather than every individual step.
    • Deep Research: Logs phase transitions such as Planning, Searching (n/total), and Synthesizing.

    Developers implementing new tools should inject the Context object and use await ctx.info("message") for key milestones.