Kindly Web Search MCP Server

repository·main·Indexed 18 days ago

https://github.com/shelpuk-ai-technology-consulting/kindly-web-search-mcp-server

An MCP server providing AI coding agents with high-fidelity web search and content retrieval. It optimizes for technical documentation and developer discussions by retrieving full conversations from StackOverflow, GitHub, and arXiv, or parsing webpages via a headless Chromium browser. It features two primary tools: web_search and get_content, and supports multiple search providers including Serper, SerpBase, Tavily, SearXNG, and Sofya.

Tokens
16.8K
Snippets
48
Records
67
Agent score
62%

What's inside kindly-web-search-mcp-server

  1. How the Kindly Web Search MCP Server works

    main

    Kindly Web Search is an MCP server designed to provide AI coding assistants with up-to-date API documentation and robust web content retrieval. Unlike generic search MCP servers that only return snippets, Kindly retrieves full conversations (questions, answers, comments, and metadata) from specialized sources like StackOverflow, GitHub, and arXiv, or parses arbitrary webpages using a headless browser.

    It uses a priority-based content resolver to fetch data efficiently:

    1. StackExchange API (StackOverflow, etc.)
    2. GitHub GraphQL API (Issues and Discussions)
    3. MediaWiki Action API (Wikipedia)
    4. Atom API + PDF to Markdown (arXiv)
    5. Universal HTML loader (Headless Chromium via nodriver for all other URLs)

    It also features 'Markdown fast paths' to skip the browser for documentation sites that support direct Markdown fetching via specific host suffixes or Accept: text/markdown headers.

  2. Configure Kindly Web Search for Cursor

    main

    To use the Kindly Web Search MCP server with Cursor, create a .cursor/mcp.json file in your project.

    Note on Timeouts

    Cursor does not currently support per-server startup timeouts. If the initial run is slow, run the uvx command from the Quickstart section once in a terminal to prebuild the environment, then restart Cursor.

    {
      "mcpServers": {
        "kindly-web-search": {
          "type": "stdio",
          "command": "uvx",
          "args": [
            "--from",
            "git+https://github.com/Shelpuk-AI-Technology-Consulting/kindly-web-search-mcp-server",
            "kindly-web-search-mcp-server",
            "start-mcp-server"
          ],
          "env": {
            "SERPER_API_KEY": "${env:SERPER_API_KEY}",
            "TAVILY_API_KEY": "${env:TAVILY_API_KEY}",
            "SEARXNG_BASE_URL": "${env:SEARXNG_BASE_URL}",
            "GITHUB_TOKEN": "${env:GITHUB_TOKEN}",
            "KINDLY_BROWSER_EXECUTABLE_PATH": "${env:KINDLY_BROWSER_EXECUTABLE_PATH}"
          }
        }
      }
    }
  3. Deploy the MCP server via Docker (Streamable HTTP)

    main

    If your MCP client supports HTTP connections, you can run the server on a remote machine or in a container using Streamable HTTP.

    Deployment Steps:

    1. Build the image: docker build -t kindly-web-search-mcp-server .
    2. Run the container with the --http flag, specifying a host and port.
    3. Ensure at least one search provider key (e.g., SERPER_API_KEY, TAVILY_API_KEY) is provided via environment variables.

    Endpoint: http://<server-host>:8000/mcp

    Security Warning: Remote HTTP is unauthenticated and unencrypted by default. Do not expose this port publicly without a VPN, firewall, or reverse proxy with TLS and authentication.

    docker run --rm -p 8000:8000 \
      -e SERPER_API_KEY="..." \
      -e GITHUB_TOKEN="..." \
      -e KINDLY_CHROME_PROXY="socks5://host.docker.internal:1080" \
      kindly-web-search-mcp-server \
      --http --host 0.0.0.0 --port 8000
  4. Install a Chromium-based browser for `page_content` extraction

    main

    To use the page_content tool for universal extraction from arbitrary websites, you must have a Chromium-based browser (Chrome, Chromium, Edge, or Brave) installed on the same machine running the MCP client.

    If you skip this, specialized sources like StackOverflow, GitHub, Wikipedia, and arXiv will still function, but general web scraping will be limited.

    macOS:

    brew install --cask chromium

    Windows: Install Chrome or Edge. If auto-detection fails, you may need to provide the path to the executable (e.g., C:\Program Files\Google\Chrome\Application\chrome.exe).

    Linux (Ubuntu/Debian):

    sudo apt-get update
    sudo apt-get install -y chromium
  5. Configure Kindly Web Search for OpenClaw

    main

    To use the Kindly Web Search MCP server with OpenClaw, it is recommended to use mcporter via the CLI.

    CLI Installation

    Use mcporter config add to add the server. This writes to ~/.mcporter/mcporter.json when using --scope home.

    File-based Configuration

    Alternatively, edit your mcporter.json file directly.

    Warning: Do not add mcpServers to the root level of ~/.openclaw/openclaw.json, as OpenClaw uses strict schema validation and will reject unknown keys. If the server is not detected, run openclaw gateway restart.

    Verification

    Verify the configuration using mcporter config get <server-name>.

    # Install mcporter if not present
    npm i -g mcporter
    
    # Add server via CLI
    mcporter config add kindly-search \
      --scope home \
      --command "uvx --from git+https://github.com/Shelpuk-AI-Technology-Consulting/kindly-web-search-mcp-server kindly-web-search-mcp-server start-mcp-server" \
      --env SERPER_API_KEY="$SERPER_API_KEY" \
      --env TAVILY_API_KEY="$TAVILY_API_KEY" \
      --env SEARXNG_BASE_URL="$SEARXNG_BASE_URL" \
      --env GITHUB_TOKEN="$GITHUB_TOKEN" \
      --env KINDLY_BROWSER_EXECUTABLE_PATH="$KINDLY_BROWSER_EXECUTABLE_PATH"
  6. Configure Kindly Web Search for Claude Desktop

    main

    To use the Kindly Web Search MCP server with Claude Desktop, edit your claude_desktop_config.json file.

    File Locations:

    • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
    • Windows: %APPDATA%\Claude\claude_desktop_config.json

    Important Notes

    • Values in this file are literal strings. Do not commit or share this file.
    • Claude Desktop does not support per-server startup timeouts. If the first run is slow, run the uvx command from the Quickstart section once in a terminal to prebuild the environment, then restart Claude Desktop.
    {
      "mcpServers": {
        "kindly-web-search": {
          "command": "uvx",
          "args": [
            "--from",
            "git+https://github.com/Shelpuk-AI-Technology-Consulting/kindly-web-search-mcp-server",
            "kindly-web-search-mcp-server",
            "start-mcp-server"
          ],
          "env": {
            "SERPER_API_KEY": "PASTE_SERPER_KEY_OR_LEAVE_EMPTY",
            "TAVILY_API_KEY": "PASTE_TAVILY_KEY_OR_LEAVE_EMPTY",
            "SEARXNG_BASE_URL": "PASTE_SEARXNG_URL_OR_LEAVE_EMPTY",
            "GITHUB_TOKEN": "PASTE_GITHUB_TOKEN_OR_LEAVE_EMPTY",
            "KINDLY_BROWSER_EXECUTABLE_PATH": "PASTE_IF_NEEDED"
          }
        }
      }
    }
  7. Configure Kindly Web Search for Antigravity (Google IDE)

    main

    To add the Kindly Web Search MCP server to Antigravity:

    1. Open the MCP store in Antigravity.
    2. Click Manage MCP Servers.
    3. Click View raw config to open mcp_config.json.
    4. Add the server configuration under the mcpServers object.
    5. Save the file and click Refresh.

    Troubleshooting

    • If uvx is not found, use its absolute path (e.g., via which uvx on macOS/Linux or where uvx on Windows).
    • If the first startup is slow, run the uvx command from the Quickstart section once in a terminal to prebuild the environment, then click Refresh.
    • Security Note: Do not commit or share mcp_config.json if it contains sensitive API keys.
    {
      "kindly-web-search": {
        "command": "uvx",
        "args": [
          "--from",
          "git+https://github.com/Shelpuk-AI-Technology-Consulting/kindly-web-search-mcp-server",
          "kindly-web-search-mcp-server",
          "start-mcp-server"
        ],
        "env": {
          "SERPER_API_KEY": "PASTE_SERPER_KEY_OR_LEAVE_EMPTY",
          "TAVILY_API_KEY": "PASTE_TAVILY_KEY_OR_LEAVE_EMPTY",
          "SEARXNG_BASE_URL": "PASTE_SEARXNG_URL_OR_LEAVE_EMPTY",
          "GITHUB_TOKEN": "PASTE_GITHUB_TOKEN_OR_LEAVE_EMPTY",
          "KINDLY_BROWSER_EXECUTABLE_PATH": "PASTE_IF_NEEDED"
        }
      }
    }
  8. Configure Kindly Web Search for Gemini CLI

    main

    To use the Kindly Web Search MCP server with Gemini CLI, edit your ~/.gemini/settings.json (or .gemini/settings.json in your project directory). You must provide one of the search provider credentials: SERPER_API_KEY, SERPBASE_API_KEY, TAVILY_API_KEY, SEARXNG_BASE_URL, or SOFYA_API_KEY.

    {
      "mcpServers": {
        "kindly-web-search": {
          "command": "uvx",
          "args": [
            "--from",
            "git+https://github.com/Shelpuk-AI-Technology-Consulting/kindly-web-search-mcp-server",
            "kindly-web-search-mcp-server",
            "start-mcp-server"
          ],
          "env": {
            "SERPER_API_KEY": "$SERPER_API_KEY",
            "TAVILY_API_KEY": "$TAVILY_API_KEY",
            "SEARXNG_BASE_URL": "$SEARXNG_BASE_URL",
            "GITHUB_TOKEN": "$GITHUB_TOKEN",
            "KINDLY_BROWSER_EXECUTABLE_PATH": "$KINDLY_BROWSER_EXECUTABLE_PATH"
          },
          "timeout": 120000
        }
      }
    }
  9. Configure Kindly Web Search for GitHub Copilot / VS Code

    main

    The most secure way to configure the Kindly Web Search MCP server in VS Code (GitHub Copilot / Microsoft Copilot) is to use interactive prompts. This avoids storing secrets in plain text files.

    Create a .vscode/mcp.json file in your project and define the server and its required inputs. This will trigger prompts for the API keys and other configuration values when the server starts.

    {
      "servers": {
        "kindly-web-search": {
          "type": "stdio",
          "command": "uvx",
          "args": [
            "--from",
            "git+https://github.com/Shelpuk-AI-Technology-Consulting/kindly-web-search-mcp-server",
            "kindly-web-search-mcp-server",
            "start-mcp-server"
          ],
          "env": {
            "SERPER_API_KEY": "${input:serper-api-key}",
            "TAVILY_API_KEY": "${input:tavily-api-key}",
            "SEARXNG_BASE_URL": "${input:searxng-base-url}",
            "GITHUB_TOKEN": "${input:github-token}",
            "KINDLY_BROWSER_EXECUTABLE_PATH": "${input:browser-path}"
          }
        }
      },
      "inputs": [
        { "id": "serper-api-key", "type": "promptString", "description": "Serper API key (optional if using Tavily or SearXNG)" },
        { "id": "tavily-api-key", "type": "promptString", "description": "Tavily API key (optional if using Serper or SearXNG)" },
        { "id": "searxng-base-url", "type": "promptString", "description": "SearXNG base URL (optional if using Serper or Tavily)" },
        { "id": "github-token", "type": "promptString", "description": "GitHub token (recommended)" },
        { "id": "browser-path", "type": "promptString", "description": "Browser binary path (only if needed)" }
      ]
    }
  10. Configure Kindly Web Search for Claude Code

    main

    To use the Kindly Web Search MCP server with Claude Code, provide one of the search provider credentials: SERPER_API_KEY, SERPBASE_API_KEY, TAVILY_API_KEY, SEARXNG_BASE_URL, or SOFYA_API_KEY.

    CLI Installation

    Use the claude mcp add command. Note that for current Claude Code versions, the server name must be placed immediately after --transport stdio and before any -e or --env flags.

    File-based Configuration

    Create or edit a .mcp.json file (recommended for project-specific scope and teams).

    Handling Timeouts

    If Claude Code times out during startup, increase the timeout by setting the MCP_TIMEOUT environment variable (value in milliseconds).

    # macOS / Linux (Serper example)
    claude mcp add --transport stdio kindly-web-search \
      -e SERPER_API_KEY="$SERPER_API_KEY" \
      -e GITHUB_TOKEN="$GITHUB_TOKEN" \
      -e KINDLY_BROWSER_EXECUTABLE_PATH="$KINDLY_BROWSER_EXECUTABLE_PATH" \
      -- uvx --from git+https://github.com/Shelpuk-AI-Technology-Consulting/kindly-web-search-mcp-server \
      kindly-web-search-mcp-server start-mcp-server
    
    # Increase timeout if needed
    export MCP_TIMEOUT=120000