MagicTunnel

MagicTunnel is an intelligent agent orchestration bridge that provides a unified interface for managing multiple agent types and external Model Context Protocol (MCP) servers.

Key functional areas include:

Agent Routing

Supports multiple agent types including Subprocess, HTTP, LLM, and WebSocket agents. It features Handlebars-style templating for parameter substitution and includes built-in timeout management and retry logic.

MCP Specification Compliance

• Resource Management: Read-only resource system with URI validation.
• Prompt Templates: Management of templates with argument substitution.
• Logging: RFC 5424 syslog-compliant logging with 8 severity levels. Log levels can be controlled dynamically via the /mcp/logging/setLevel HTTP endpoint.
• Notifications: Event-driven system with resource subscriptions.
• Communication: Full JSON-RPC 2.0 message handling via WebSockets and gRPC streaming.

MCP Proxy Integration

Allows connecting to external MCP servers via a WebSocket-based client. It includes a registry system for managing multiple connections, intelligent tool name mapping to resolve conflicts, and tool aggregation to provide a single unified interface for both local and remote tools.

Capability Generation

• GraphQL: 100% specification compliance, including SDL and introspection JSON parsing.
• OpenAPI: Supports OpenAPI 3.0 and Swagger 2.0 (JSON/YAML). It handles full $ref resolution, complex nested objects, and various authentication methods (API Key, Bearer, Basic, OAuth).

Smart Discovery is a natural language interface for MagicTunnel that allows you to find and execute tools without knowing their specific names or parameter schemas. Instead of calling a specific tool, you provide a natural language request describing your goal, and the system discovers and executes the appropriate tool.

Core Request Structure

When calling the smart_tool_discovery tool, you provide an arguments object containing:

• request: A string describing the task (e.g., "read the config.yaml file").
• context (optional): Additional details to improve discovery accuracy.
• preferred_tools (optional): A list of tool names to prioritize.
• confidence_threshold (optional): A float to control how strictly the system matches tools.
• include_error_details (optional): A boolean to enable verbose error reporting.

{
  "name": "smart_tool_discovery",
  "arguments": {
    "request": "read the config.yaml file"
  }
}

MagicTunnel sits between an MCP Client and your Agents & Endpoints.

1. MCP Client: Any standard-compliant client (e.g., Claude Desktop).
2. MagicTunnel: The central gateway that manages a Capability Registry and performs Agent Routing.
3. Agents & Endpoints: The actual execution layer, which includes:
   • Custom Agents: Local subprocesses, HTTP services, gRPC, GraphQL, LLMs, etc.
   • External MCP Servers: Existing MCP servers managed and proxied by MagicTunnel.

This architecture allows for seamless tool discovery and conflict resolution between local custom agents and remote MCP servers.

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   MCP Client    │───▶│  MagicTunnel    │───▶│  Agents &       │
│  (Any Client)   │    │  (This Project) │    │  Endpoints      │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                              │                        │
                              ▼                        ▼
                       ┌─────────────────┐    ┌─────────────────┐
                       │ Capability      │    │ External MCP    │
                       │ Registry        │    │ Servers         │
                       └─────────────────┘    └─────────────────┘

MagicTunnel uses four distinct strategies to find the right tool for a request:

1. Rule-based Search (Fast): Uses exact tool name matching, fuzzy fallback, keyword matching in descriptions, and category-based classification (e.g., network, file, database).
2. Semantic Search (Intelligent): Uses vector similarity (embeddings) to match conceptual requests.
   • Supported Models: OpenAI (text-embedding-3-small, text-embedding-3-large), Ollama (nomic-embed-text), and fallbacks like all-MiniLM-L6-v2.
3. LLM-based Search (Advanced): Uses AI models (OpenAI, Anthropic, Ollama) to perform complex reasoning and disambiguation.
4. Hybrid Search (Recommended): A three-layer approach combining all three methods for production environments. It uses a weighted scoring system:
   • Semantic Search: 30% weight
   • Rule-Based Matching: 15% weight
   • LLM Intelligence: 55% weight

Processing Flow: User Request $\rightarrow$ Semantic Search $\rightarrow$ Rule-Based Search $\rightarrow$ LLM Evaluation $\rightarrow$ Combined Scoring $\rightarrow$ Tool Selection.

MagicTunnel acts as a strategic gateway within the Model Context Protocol (MCP) ecosystem by performing a dual role: it functions as both an MCP Server and an MCP Client (Proxy).

Dual Role Architecture

1. As an MCP Server (Primary Role): MagicTunnel aggregates capabilities from multiple sources (such as non-MCP agents like Bash or APIs) into a single, unified MCP server. An MCP Client (like Claude or GPT) connects to MagicTunnel to access this aggregated Capability Registry.
2. As an MCP Client (Proxy Role): MagicTunnel can act as a proxy for other existing MCP servers. It connects to external MCP servers (e.g., GitHub, filesystem, or database servers) and exposes their tools to the end-user's MCP Client through its own interface.

This creates a hierarchical MCP architecture where MagicTunnel sits between the primary AI client and a variety of specialized tool providers.