web-eval-agent

To manually add the web-eval-agent to your IDE (Cursor, Cline, or Windsurf), add the following JSON configuration. Replace <YOUR_KEY> with your actual API key from operative.sh/mcp.

{
  "web-eval-agent": {
    "command": "uvx",
    "args": [
      "--refresh-package",
      "webEvalAgent",
      "--from",
      "git+https://github.com/Operative-Sh/web-eval-agent.git",
      "webEvalAgent"
    ],
    "env": {
      "OPERATIVE_API_KEY": "<YOUR_KEY>"
    }
  }
}

If you are not receiving updates in your code editor, you can force a refresh by cleaning the uv cache and refreshing the MCP server. Use the following command to ensure you are using the latest version from the repository:

uv cache clean

Ensure your configuration uses the --refresh-package flag with uvx as shown in the configuration guide.

The web_eval_agent tool is an automated UX evaluator that drives a browser, captures screenshots, console logs, and network logs to return a rich UX report. You can trigger this tool directly from your IDE chat.

Arguments:

• url (required): The address of the running application (e.g., http://localhost:3000).
• task (required): A natural-language description of the test to perform (e.g., "run through the signup flow and note any UX issues").
• headless_browser (optional, default false): Set to true to hide the browser window.

Evaluate my app at http://localhost:3000 – run web_eval_agent with the task "Try the full signup flow and report UX issues".

The MCP server requires a valid OperativeAI API key to function. This key must be provided via the OPERATIVE_API_KEY environment variable.

If the key is missing or invalid (e.g., free tier limit reached), the tools will return an error message directing you to subscribe at https://operative.sh.

The web-eval-agent determines which backend to connect to using the USE_LOCAL_BACKEND environment variable.

• To use a local backend (e.g., running on your own machine), set USE_LOCAL_BACKEND=true in your .env file or environment. This defaults the base URL to http://0.0.0.0:8000.
• To use the hosted production backend, ensure USE_LOCAL_BACKEND is either unset or not set to true. This defaults the base URL to https://operative-backend.onrender.com.

The agent automatically loads variables from a .env file if it exists in the current directory.

The get_backend_url function constructs the full URL for the backend service. It handles base URL selection based on the USE_LOCAL_BACKEND environment variable and allows appending an optional path.

Arguments:

• path (str, optional): An optional path to append to the base URL. The function automatically handles leading/trailing slashes to ensure a valid URL structure.

Returns:

• str: The complete, formatted backend URL.

The setup_browser_state tool is used to launch a non-headless browser to allow for manual user interaction, such as logging in or performing authentication. Once the interaction is complete, the tool saves the browser state (cookies, local storage, etc.) to a local file for future use.

When to use: Use this tool only when the user explicitly requests to set up browser state or authentication.

Arguments:

• url (Optional): The URL to navigate to upon opening the browser.
• ctx: The MCP context (used for progress reporting).