DOCSEARCHbyAlgolia
Submit Docs

MCP DevTools

repository·main·Indexed 41 days ago

https://github.com/sammcj/mcp-devtools

A high-performance, unified Model Context Protocol (MCP) server written in Go. It provides a comprehensive suite of developer tools including search, intelligence, and utilities in a single binary, featuring support for OAuth 2.0/2.1 authentication and OpenTelemetry-based observability with Jaeger, Prometheus, and Grafana.

Tokens
93.1K
Snippets
262
Records
414
Agent score
48%

What's inside MCP DevTools

  1. Overview of MCP DevTools

    main

    MCP DevTools is a high-performance, single Go binary that serves as a unified Model Context Protocol (MCP) server. It is designed to replace multiple resource-heavy Node.js and Python-based MCP servers by providing a modular interface for essential developer tools.

    Key Benefits

    • Single Binary Solution: Replaces multiple separate servers with one efficient binary, reducing memory footprint and startup times.
    • Comprehensive Tool Suite: Provides over 20 essential developer agent tools in one package.
    • Consistent Performance: Built in Go for speed, efficiency, and minimal resource usage.
    • Observability: Includes OpenTelemetry support for tracing and metrics.
    • Modular Design: Uses a tool registry that allows for easy extension with new tools.

  2. Overview of OpenTelemetry integration in MCP DevTools

    main

    MCP DevTools integrates OpenTelemetry (OTEL) to provide distributed tracing, performance analysis, and workflow debugging. This integration is designed to be optional and zero-overhead, meaning it has no performance impact when not configured. It is particularly useful for production HTTP deployments, measuring external API latency, and tracing complex AI agent workflows (e.g., Claude, Kiro, Gemini) that call multiple tools in sequence.

    Key capabilities include:

    • End-to-end request tracing: Tracking requests across tool chains.
    • Latency breakdown: Identifying bottlenecks in external API calls.
    • Session-based correlation: Correlating issues across concurrent clients and multi-tool workflows.
    • Standardized observability: Using OTLP (OpenTelemetry Protocol) for vendor-neutrality (compatible with Jaeger, Honeycomb, Datadog, etc.).

  3. What is Traced in mcp-devtools?

    main

    Tracing provides request-level detail for specific operations:

    • Tool Execution: Every tool invocation creates a span containing the tool name, sanitized arguments, success/failure status, session ID, and duration.
    • HTTP Requests: All HTTP requests to external APIs (e.g., via internet search or GitHub tools) create child spans with method, sanitized URL, status code, response size, and target host.
    • Security Framework (Opt-in): When MCP_TRACING_SECURITY_ENABLED=true, spans are created for content analysis, including content size, security action (allow/warn/block), and matched risk factors.
    • W3C Trace Context: The HTTP transport automatically extracts and propagates traceparent and tracestate headers, enabling distributed tracing across systems.

  4. Optimize `search_packages` performance

    main

    To ensure efficient usage of the package search tool, follow these best practices:

    1. Use batch operations: It is more efficient to query multiple packages in one call than individual queries.
    2. Specify version constraints: Providing specific versions reduces the processing time required to resolve dependencies.
    3. Use appropriate limits: Use the limit parameter (where available, e.g., in docker) to avoid retrieving unnecessary data.
    4. Cache results: Since package versions change infrequently, caching responses can significantly improve performance.

  5. How to create new tools in MCP DevTools

    main

    The MCP DevTools server is extensible. To add a new tool, you must implement the tools.Tool interface and register it with the internal registry. Tools are built using the internal/mcpapi package, which provides a builder-style wrapper around the official modelcontextprotocol/go-sdk.

    Key Concepts:

    • mcpapi vs SDK: Use mcpapi for tool definitions (e.g., mcpapi.NewTool, mcpapi.WithString). Use the official SDK directly only for server-level features like middleware or resources in main.go.
    • Input Schemas: Input schemas must follow JSON Schema draft 2020-12. mcpapi.NewTool automatically produces the object schema required by the MCP spec.
    • Tool Lifecycle: A tool is defined by its Definition() (metadata and parameters) and its Execute() (the actual logic).
    // A typical tool implementation follows this structure:
// 1. Define a struct for the Tool
// 2. Register the tool in an init() function
// 3. Implement Definition() for MCP registration
// 4. Implement Execute() for the tool's logic

  6. How provider fallback works

    main

    The tool uses an automatic fallback mechanism to ensure reliability. If a configured provider fails (due to rate limits or network errors), the tool waits with progressive delays (1s, 2s, 3s) and retries with the next available provider in the chain. Unconfigured providers are excluded from the chain.

    Fallback Order (if all are configured): Brave → Google → Kagi → SearXNG → DuckDuckGo

  7. How the MCP DevTools Security Framework works

    main

    The security framework acts as a multi-layered defense mechanism between MCP tools (like Filesystem, WebFetch, or GitHub tools) and the AI agent. It provides three main layers of protection:

    1. Access Control: Prevents tools from accessing sensitive files (e.g., ~/.ssh/id_rsa) or untrusted domains.
    2. Content Analysis: Scans returned content for security threats using pattern matching (Regex, Glob, Entropy, etc.).
    3. Lightweight Event Audit Logging: Tracks all security events for review.

    Action Processing Flow

    When a tool is called, the Security Middleware makes a decision:

    • Allow: Content is returned to the agent.
    • Warn: A warning is logged, but content is still returned to the agent.
    • Block: The request is blocked, and a Security ID is returned. This ID can be used with the security_override tool to bypass the block if authorized.

  8. Compare PDF Processing vs Document Processing

    main

    Choose the appropriate tool based on your requirements:

    FeaturePDF ProcessingDocument Processing
    Speed⚡ Fast (1-15 seconds)🐌 Slower (10-60+ seconds)
    File TypesPDF onlyPDF, DOCX, XLSX, PPTX, HTML, CSV, images
    OCR Support❌ No✅ Yes
    Diagram Analysis❌ No✅ Yes with AI models
    Setup Complexity✅ Simple⚙️ Complex
    Resource Usage🟢 Low🟡 Moderate to High

  9. How MCP DevTools OAuth 2.0/2.1 modes work

    main

    MCP DevTools supports two distinct OAuth 2.0/2.1 modes for HTTP-based transports, depending on your deployment environment:

    1. Browser Authentication Mode (OAuth Client): Designed for development and desktop environments. It uses an authorization code flow with PKCE, automatically launching a browser for interactive user login and handling localhost callbacks.
    2. Resource Server Mode (OAuth Token Validation): Designed for production API servers. It validates incoming JWT tokens from external clients by fetching JWKS (JSON Web Key Sets) from the provider to verify signatures, claims, audience, and expiration.

    You can also run both modes simultaneously for complex multi-tenant systems where the server needs its own authentication while also validating external client tokens.

    # Example: Running both modes together
./bin/mcp-devtools --transport=http --port=18080 \
    --oauth-browser-auth \
    --oauth-client-id="your-client-id" \
    --oauth-issuer="https://your-auth-server/application/o/mcpoauth/" \
    --oauth-enabled \
    --oauth-audience="http://localhost:18080" \
    --oauth-jwks-url="https://your-auth-server/application/o/mcpoauth/jwks/"

  10. Understand the OpenTelemetry architecture in MCP DevTools

    main

    MCP DevTools uses OpenTelemetry (OTEL) to provide observability into tool executions, LLM calls, and system operations. The architecture follows a layered approach:

    1. Tracer Provider: Initialized from environment variables (disabled by default).
    2. Transports: Supports stdio, HTTP (W3C TC), and SSE transports.
    3. Middleware: A Tool Execution Middleware creates a span for every tool invocation.
    4. Instrumentation: Spans are generated across various layers including HTTP clients, Security frameworks, Agent calls, Cache operations, and Database queries.
    5. Exporter: Uses an OTLP Exporter (gRPC or HTTP) to send data to an OTEL Collector.
    6. Backend: Data can be visualized in backends such as Jaeger, Langfuse, Grafana Tempo, Honeycomb, or Datadog.
    ┌─────────────────────────────────────────────────────────────┐
│                     MCP DevTools Binary                      │
│                                                              │
│  ┌────────────────────────────────────────────────────────┐ │
│  │              OTEL Tracer Provider                       │ │
│  │  (Initialised from environment, disabled by default)    │ │
│  └────────────────────────────────────────────────────────┘ │
│                           │                                  │
│                           ▼                                  │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐    │
│  │   stdio     │    │    HTTP     │    │     SSE     │    │
│  │  Transport  │    │  Transport  │    │  Transport  │    │
│  │             │    │   (W3C TC)  │    │             │    │
│  └─────────────┘    └─────────────┘    └─────────────┘    │
│         │                  │                   │            │
│         └──────────────────┴───────────────────┘            │
│                           │                                  │
│                           ▼                                  │
│  ┌────────────────────────────────────────────────────────┐ │
│  │              Tool Execution Middleware                 │ │
│  │         (Creates span per tool invocation)             │ │
│  └────────────────────────────────────────────────────────┘ │
│                           │                                  │
│                           ▼                                  │
│  ┌─────────┬─────────┬─────────┬─────────┬─────────┬──────┐ │
│  │  HTTP   │ Security│  Agent  │  Cache  │   DB    │ ...  │
│  │ Client  │Framework│  Calls  │ Ops     │ Queries │      │
│  │ (auto)  │(manual) │(manual) │(manual) │ (auto)  │      │
│  └─────────┴─────────┴─────────┴─────────┴─────────┴──────┘ │
│                                                              │
└─────────────────────────────────────────────────────────────┘
                          │
                          ▼
              ┌───────────────────────┐
              │   OTLP Exporter       │
              │   (gRPC or HTTP)       │
              └───────────────────────┘
                          │
                          ▼
              ┌───────────────────────┐
              │  OTEL Collector       │
              │  (User-provided)      │
              └───────────────────────┘
                          │
                          ▼
              ┌───────────────────────┐
              │  Backend (Choice of): │
              │  - Jaeger             │
              │  - Langfuse           │
              │  - Grafana Tempo      │
              │  - Honeycomb          │
              │  - Datadog            │
              │  - etc.               │
              └───────────────────────┘

  11. Understand PDF Processing output structure

    main

    When processing a PDF, the tool generates a Markdown file and an images directory. The structure follows this pattern:

    /path/to/document.pdf
├── document.md          # Extracted markdown content
└── images/              # Extracted images directory
    ├── image_001.png
    ├── image_002.jpg
    └── ...

    The generated document.md uses standard Markdown syntax for images, linking them to the local images/ directory (e.g., ![Image 1](images/image_001.png)).

  12. OpenTelemetry design principles and constraints

    main

    The OpenTelemetry implementation follows four core principles to ensure it remains lightweight and secure:

    1. Optional & Zero-Overhead: Tracing is disabled by default and only activates when OTEL_EXPORTER_OTLP_ENDPOINT is provided. The system degrades gracefully if the collector is unavailable.
    2. Single Binary Philosophy: All OTEL functionality is compiled into the main mcp-devtools binary; no additional binaries are required.
    3. Standards Compliance: Uses W3C Trace Context for HTTP header propagation and follows OTLP (gRPC or HTTP) and standard OTEL semantic conventions.
    4. Privacy & Security: The system is designed to never trace sensitive data such as credentials, tokens, or API keys. It also supports configurable sampling to manage high-volume scenarios.