Bright Data MCP Server

repository·main·Indexed 25 days ago

https://github.com/brightdata/brightdata-mcp

A Model Context Protocol (MCP) server that provides LLMs with real-time web access, scraping capabilities, and structured datasets from sources like e-commerce, social media, and npm. It supports three operational modes: Rapid (free), Pro (pay-as-you-go with 60+ tools), and Custom (whitelist-based). The server integrates with MCP clients such as Claude Desktop and supports specialized tool groups for browser automation, finance, business intelligence, and coding agents.

Tokens
5.3K
Snippets
10
Records
28
Agent score
83%

What's inside @brightdata/mcp

  1. Understand Bright Data MCP Pricing and Modes

    main

    The MCP server operates in three distinct modes controlled by environment variables:

    1. Rapid Mode (Default): The free tier. Includes 5,000 credits/month (auto-renewing). Supports Web Search, Scraping with Web unlocker, and AI-ranked Discover search. Does not include Browser Automation or Web data tools.
    2. Pro Mode: Pay-as-you-go. Includes everything in Rapid mode plus 60+ tools like Browser Control and Web Data APIs. Enable via PRO_MODE=true.
    3. Custom Mode: Usage-based. Allows you to whitelist specific tool groups or individual tools. Enable via GROUPS or TOOLS environment variables. Note that using GROUPS or TOOLS overrides Pro eligibility.

    Credit Usage:

    • Base tools (search_engine, scrape_as_markdown, discover) cost 1 credit per request.
    • web_data_* tools cost 1 credit per record returned.
  2. Explore Bright Data MCP community examples

    main

    The examples/README.md file provides a curated list of community-driven implementations and demos using the Bright Data MCP server. These examples demonstrate various high-level use cases including:

    • AI Voice Agents: Automating sales and deal closing.
    • Agentic Frameworks: Integration with Langgraph (via mcp-adapters) and Google ADK for researcher agents.
    • Web Scraping: Real-time scraping of any website while avoiding blocks.
    • Multi-Agent Systems: Building complex workflows like job finders using TypeScript.
    • CLI Integration: Using the server with the Gemini CLI.

    For developers looking to build specialized agents, these examples serve as architectural references for connecting real-time data fetching to LLM-driven workflows.

  3. Quick Start: Use the hosted Bright Data MCP server

    main

    For zero-configuration setup, you can use the hosted MCP server directly by adding the following URL to your MCP client (e.g., Claude Desktop):

    https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN_HERE

    Setup in Claude Desktop:

    1. Go to: SettingsConnectorsAdd custom connector
    2. Name: Bright Data Web
    3. URL: https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN
    4. Click "Add".
    https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN_HERE
  4. Configure the Bright Data MCP Server

    main

    To use the Bright Data MCP server in an MCP client (like Claude Desktop), add it to your mcpServers configuration. You must provide a valid API_TOKEN via environment variables.

    Basic Setup

    Use this configuration for standard web search and markdown scraping capabilities.

    Advanced Configuration

    Enable PRO_MODE to access over 60+ specialized tools (Browser automation, E-commerce, Social Media, etc.) and customize performance settings like rate limits and timeouts.

    {
      "mcpServers": {
        "Bright Data": {
          "command": "npx",
          "args": ["@brightdata/mcp"],
          "env": {
            "API_TOKEN": "your-token-here",
            "PRO_MODE": "true",              // Enable all 60+ tools
            "RATE_LIMIT": "100/1h",          // Custom rate limiting
            "WEB_UNLOCKER_ZONE": "custom",   // Custom unlocker zone
            "BROWSER_ZONE": "custom_browser", // Custom browser zone
            "POLLING_TIMEOUT": "600"         // Polling timeout in seconds (default: 600)
          }
        }
      }
    }
  5. Quick Start: Run Bright Data MCP locally

    main

    To run the MCP server locally on your machine using npx, add the following configuration to your MCP settings file:

    {
      "mcpServers": {
        "Bright Data": {
          "command": "npx",
          "args": ["@brightdata/mcp"],
          "env": {
            "API_TOKEN": "<your-api-token-here>"
          }
        }
      }
    }
    {
      "mcpServers": {
        "Bright Data": {
          "command": "npx",
          "args": ["@brightdata/mcp"],
          "env": {
            "API_TOKEN": "<your-api-token-here>"
          }
        }
      }
    }
  6. Run evaluations using mcpjam CLI

    main

    To execute the tests, use the mcpjam evals run command with the following flags:

    • -t <tool-groups-file>: Path to the tool group test definition file.
    • -e <server-config>: Path to the server configuration file.
    • -l <llms-config>: Path to the LLM API keys configuration file.

    Example command for E-commerce:

    mcpjam evals run \
      -t tool-groups.json/tool-groups.ecommerce.json \
      -e server-configs/server-config.ecommerce.json \
      -l llms.json
  7. Define Tool Group Test cases in tool-groups.json

    main

    Test cases are defined in JSON files within the tool-groups.json/ directory. Each test case object specifies the parameters for an automated evaluation run.

    Fields:

    • title: Description of the test.
    • query: The natural language prompt sent to the LLM.
    • runs: Number of times to execute the test.
    • model: The specific LLM model string (e.g., gpt-5.1-2025-11-13).
    • provider: The LLM provider (e.g., openai).
    • expectedToolCalls: An array of tool names that the LLM is expected to invoke.
    • selectedServers: An array of server names defined in your server configuration.
    • advancedConfig: Optional object containing:
      • instructions: System instructions for the LLM.
      • temperature: LLM temperature.
      • maxSteps: Maximum number of tool call steps.
      • toolChoice: Strategy for tool selection (required or auto).

    Example test case:

    {
      "title": "Test E-commerce - Amazon product search",
      "query": "Search for wireless headphones on Amazon and show me the top products with reviews",
      "runs": 1,
      "model": "gpt-5.1-2025-11-13",
      "provider": "openai",
      "expectedToolCalls": ["web_data_amazon_product_search"],
      "selectedServers": ["ecommerce-server"],
      "advancedConfig": {
        "instructions": "You are a shopping assistant helping users find products on Amazon",
        "temperature": 0.1,
        "maxSteps": 5,
        "toolChoice": "required"
      }
    }
  8. Configure Bright Data MCP Server settings

    main

    Server configuration files (located in server-configs/*.json) define how the evaluation framework connects to the Bright Data MCP server. Each file specifies a timeout (in milliseconds) and a servers object containing connection details.

    Key fields:

    • timeout: Maximum execution time for the test.
    • url: The MCP server endpoint, which should include groups and browser parameters in the query string.
    • requestInit.headers: Used for authentication (e.g., Authorization: Bearer <token>).

    Example server-config.ecommerce.json:

    {
      "timeout": 600000,
      "servers": {
        "ecommerce-server": {
          "url": "http://mcp.brightdata.com/mcp?groups=ecommerce&browser=scraping_browser",
          "requestInit": {
            "headers": {
              "Authorization": "Bearer YOUR_AUTH_TOKEN"
            }
          }
        }
      }
    }
  9. Configure Advanced Tool Selection with GROUPS and TOOLS

    main

    You can customize the available tools by using environment variables in your MCP configuration:

    • GROUPS: Enables curated tool bundles. Use comma-separated IDs (e.g., ecommerce,browser).
    • TOOLS: Adds specific tool names on top of selected groups (e.g., extract).

    Mode Priority: PRO_MODE=true (all tools) → GROUPS / TOOLS (whitelist) → default rapid mode (base toolkit).

    Available Group IDs:

    • ecommerce: Retail/marketplace (e.g., web_data_amazon_product)
    • social: Social/community (e.g., web_data_linkedin_posts)
    • browser: Scraping Browser automation (e.g., scraping_browser_screenshot)
    • finance: Financial intelligence (e.g., web_data_yahoo_finance_business)
    • business: Company/location intelligence (e.g., web_data_crunchbase_company)
    • research: News/developer feeds (e.g., web_data_github_repository_file)
    • app_stores: App store data (e.g., web_data_google_play_store)
    • travel: Travel info (e.g., web_data_booking_hotel_listings)
    • geo: GEO & LLM brand visibility (e.g., web_data_chatgpt_ai_insights)
    • code: Package intelligence (e.g., web_data_npm_package)
    • advanced_scraping: Batch/AI extraction (e.g., scrape_batch, extract)
  10. Configure Bright Data MCP Server via Environment Variables

    main

    The Bright Data MCP server is configured using several environment variables. These variables control authentication, zone selection, tool availability, and performance settings.

    Required

    • API_TOKEN: Your Bright Data API token.

    Optional Configuration

    • WEB_UNLOCKER_ZONE: The name of the Web Unblocker zone to use (defaults to mcp_unlocker).
    • BROWSER_ZONE: The name of the Browser API zone to use (defaults to mcp_browser).
    • PRO_MODE: Set to true to enable all available tools. If not set or false, tool access is restricted by GROUPS or TOOLS.
    • GROUPS: A comma-separated list of tool group IDs to enable (e.g., group1,group2).
    • TOOLS: A comma-separated list of specific tool names to enable (e.g., search_engine,extract).
    • RATE_LIMIT: A string defining the rate limit in the format limit/time (e.g., 100/1h or 50/30m).
    • POLLING_TIMEOUT: Timeout in seconds for long-running tasks like discover (defaults to 600).
    • BASE_TIMEOUT: Base timeout for API requests in seconds.
    • BASE_MAX_RETRIES: Maximum number of retries for failed requests (capped at 3).