Firecrawl MCP Server

repository·main·Indexed 27 days ago

https://github.com/firecrawl/firecrawl-mcp-server

A Model Context Protocol (MCP) server that enables AI agents to search, scrape, and interact with the live web. It provides structured, agent-ready context from URLs and supports both cloud and self-hosted instances. Key features include web search, page interaction, batch processing, and LLM-powered content analysis via tools such as scrape, map, crawl, and extract.

Tokens
12K
Snippets
35
Records
57
Agent score
93%

What's inside firecrawl-mcp

  1. Use the Search-only MCP surface (/v2/mcp-search)

    main

    The Firecrawl MCP server provides a specialized, read-only search surface at the /v2/mcp-search endpoint. This surface is designed for restricted access and only exposes a specific subset of tools. It is distinct from the full MCP surface available at /v2/mcp.

    Key Characteristics:

    • Read-only: Only specific search and research tools are available.
    • No Page Content Fetching: The firecrawl_search tool in this surface does not support scrapeOptions. It is strictly limited to search parameters to prevent fetching third-party page content.
    • Strict Schema: Input schemas are strict; unknown fields are rejected.
    • Authentication Required: Unlike the full surface, the keyless free-tier fallback does not apply to the search surface. Every request (including tools/list) requires valid credentials.
  2. Understand Firecrawl MCP Server Versioning

    main

    The Firecrawl MCP Server supports two versions of the Firecrawl API to maintain backward compatibility while providing modern features:

    • V1 (Legacy): Based on Firecrawl JS 1.29.3. It includes extended tools like firecrawl_deep_research and firecrawl_generate_llmstxt but uses legacy parameter names.
    • V2 (Current): Based on Firecrawl JS 3.1.0. It uses modern API parameter names (e.g., ScrapeOptions, MapOptions), enhanced JSON extraction with schema support, and improved caching defaults.
  3. Choose the correct format for `scrape`

    main

    When using the scrape tool, select the output format based on your requirements:

    • JSON format (Recommended): Use this when you need specific data from a page. Define a schema to extract exactly what you need. This is the best practice to keep responses small and prevent context window overflow in LLMs.
    • Markdown format: Use this sparingly. Only select Markdown when you require the full page content, such as for full-article summarization or analyzing the overall page structure.
  4. Configure Firecrawl MCP in Windsurf

    main

    Add the following configuration to your ./codeium/windsurf/model_config.json file:

    {
      "mcpServers": {
        "mcp-server-firecrawl": {
          "command": "npx",
          "args": ["-y", "firecrawl-mcp"],
          "env": {
            "FIRECRAWL_API_KEY": "YOUR_API_KEY"
          }
        }
      }
    }
    {
      "mcpServers": {
        "mcp-server-firecrawl": {
          "command": "npx",
          "args": ["-y", "firecrawl-mcp"],
          "env": {
            "FIRECRAWL_API_KEY": "YOUR_API_KEY"
          }
        }
      }
    }
  5. Start the Firecrawl MCP Server

    main

    Depending on your deployment target, use the following commands to start the server:

    Cloud Service (Versioned Endpoints):

    npm run start:cloud

    Local Development (V2 by default):

    npm start

    SSE Local Server:

    SSE_LOCAL=true npm start

    HTTP Streamable Server:

    HTTP_STREAMABLE_SERVER=true npm start
  6. Migrate from V1 to V2

    main

    To upgrade from V1 to V2, follow these steps:

    1. Update Endpoint URLs:
      • Change /:apiKey/sse to /:apiKey/v2/sse
      • Change /:apiKey/messages to /:apiKey/v2/messages
    2. Update Tools:
      • Remove usage of firecrawl_deep_research and firecrawl_generate_llmstxt (these are V1 specific).
      • Update extraction calls to use the new schema format to leverage enhanced JSON extraction.
    3. Adjust Parameters:
      • Note that maxAge now defaults to 172800000ms (48 hours) instead of 0.
      • onlyMainContent now defaults to true.
  7. Configure Firecrawl MCP in VS Code

    main

    You can configure Firecrawl MCP in VS Code either via the UI or by manually editing your User Settings (JSON).

    Manual Configuration (User Settings JSON)

    Add this block to your settings.json (accessible via Ctrl + Shift + P > Preferences: Open User Settings (JSON)):

    {
      "mcp": {
        "inputs": [
          {
            "type": "promptString",
            "id": "apiKey",
            "description": "Firecrawl API Key",
            "password": true
          }
        ],
        "servers": {
          "firecrawl": {
            "command": "npx",
            "args": ["-y", "firecrawl-mcp"],
            "env": {
              "FIRECRAWL_API_KEY": "${input:apiKey}"
            }
          }
        }
      }
    }

    Workspace Configuration

    To share the configuration within a workspace, create a .vscode/mcp.json file:

    {
      "inputs": [
        {
          "type": "promptString",
          "id": "apiKey",
          "description": "Firecrawl API Key",
          "password": true
        }
      ],
      "servers": {
        "firecrawl": {
          "command": "npx",
          "args": ["-y", "firecrawl-mcp"],
          "env": {
            "FIRECRAWL_API_KEY": "${input:apiKey}"
          }
        }
      }
    }
    {
      "mcp": {
        "inputs": [
          {
            "type": "promptString",
            "id": "apiKey",
            "description": "Firecrawl API Key",
            "password": true
          }
        ],
        "servers": {
          "firecrawl": {
            "command": "npx",
            "args": ["-y", "firecrawl-mcp"],
            "env": {
              "FIRECRAWL_API_KEY": "${input:apiKey}"
            }
          }
        }
      }
    }
  8. Run Firecrawl MCP in Streamable HTTP Local Mode

    main

    To run the server using Streamable HTTP locally instead of the default stdio transport, use the following command and connect to the provided URL:

    Command:

    env HTTP_STREAMABLE_SERVER=true FIRECRAWL_API_KEY=fc-YOUR_API_KEY npx -y firecrawl-mcp

    URL: http://localhost:3000/mcp

    env HTTP_STREAMABLE_SERVER=true FIRECRAWL_API_KEY=fc-YOUR_API_KEY npx -y firecrawl-mcp
  9. Connect to Hosted Firecrawl MCP (Keyless or API Key)

    main

    You can connect to a remote hosted server without any local setup.

    • Keyless Free Tier: Use the endpoint below for rate-limited access to scrape, search, and interact. Other tools like crawl, map, agent, and extract require an API key.
    • With API Key: Use your Firecrawl API key to unlock the full toolset and higher limits.

    Endpoints:

    • Keyless: https://mcp.firecrawl.dev/v2/mcp
    • With API Key: https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp
    https://mcp.firecrawl.dev/v2/mcp
  10. Select the right Firecrawl tool for your task

    main

    Use the following logic to choose the appropriate tool based on your objective:

    • Single URL extraction: Use scrape. For structured data, use JSON format with a schema to keep responses small and avoid context window overflow.
    • Multiple known URLs: Call scrape for each URL individually.
    • URL discovery: Use map to find URLs on a specific site.
    • Web searching: Use search to find information across the web.
    • Complex research: Use agent for multi-source research tasks.
    • Site-wide analysis: Use crawl (be mindful of limits).
    • Browser automation: Use interact for clicking, typing, or navigating. You can use it with a URL for a fresh page, or combine scrape + interact for tighter control.
    • File/Upload processing: Use parse for files and hosted upload references.
    • Structured extraction: Use extract to get specific JSON structured data from URLs.
    • Recurring checks: Use monitor to check for page changes and metadata diffs.
    • Academic/Code research: Use research for papers and GitHub repository research.