ToolHive Documentation

repository·main·Indexed 24 days ago

https://github.com/stacklok/toolhive

An open-source platform for securely running and managing Model Context Protocol (MCP) servers. ToolHive provides isolation via containers, centralized policy enforcement, and scalable deployment for developers and enterprise Kubernetes environments. It includes a Kubernetes Operator for managing MCPServer and MCPRegistry resources, as well as the Virtual MCP Server (vmcp) for aggregating multiple MCP servers into a single instance with support for OIDC incoming authentication and RFC 8693 token exchange for outgoing authentication.

Tokens
216.4K
Snippets
430
Records
987
Agent score
84%

What's inside ToolHive

  1. Available Phase 2 features in VirtualMCPCompositeToolDefinition

    main

    The VirtualMCPCompositeToolDefinition supports advanced workflow orchestration features including:

    Completed Features

    • DAG Execution: Steps are executed in parallel where possible based on a dependency graph.
    • Step Output Access: You can reference outputs from previous steps within templates.
    • Advanced Retry Policies: Supports exponential backoff with configurable retry counts and delays.
    • Workflow State Management: Uses in-memory state tracking with a pluggable backend interface.
    • Advanced Error Handling: Supports per-step and workflow-level strategies such as abort, continue, or retry.
    • Workflow Timeouts: Configurable timeouts can be set at both the workflow and individual step levels.
    • Conditional Execution: Steps can be skipped based on template conditions.

    Planned Features (Future Releases)

    • Distributed State Store: Support for Redis or Database backends for multi-instance deployments.
    • Step Caching: Caching step results based on specific cache keys.
    • Output Transformation: Advanced transformations using templates.
    • Workflow Resumption: Ability to resume workflows after a system restart.
  2. Explore the ToolHive Architecture Documentation

    main

    The ToolHive architecture documentation is organized into several functional layers to help different user roles understand the platform.

    Documentation Structure

    • Core Understanding: High-level overview, core concepts (nouns and verbs), and deployment modes (Local vs. Kubernetes).
    • Communication Layer: Details on transport types (stdio, SSE, streamable-http) and the middleware system.
    • Configuration & Security: Information on RunConfig, permission profiles, and secrets management (encrypted, 1password, environment).
    • Distribution & Organization: Documentation on the Registry system and logical Groups.
    • Runtime Management: Lifecycle management for workloads, the Kubernetes Operator (CRDs), Virtual MCP (vMCP) aggregation, and Auth Server storage.
    • Agent Skills: The lifecycle and distribution of Agent Skills and the Plugins system.
    • Platform Developers: Start with 00-overview.md $\rightarrow$ 02-core-concepts.md $\rightarrow$ 01-deployment-modes.md.
    • Middleware Developers: Focus on 03-transport-architecture.md and the central middleware.md guide.
    • Operators: Start with 01-deployment-modes.md $\rightarrow$ 09-operator-architecture.md.
    • Contributors: It is recommended to review all documents in sequential order (00 through 16).
  3. What is a VirtualMCPCompositeToolDefinition

    main
    A VirtualMCPCompositeToolDefinition is a Kubernetes Custom Resource Definition (CRD) used to create reusable, complex workflows for Virtual MCP Servers. Instead of defining individual tool calls repeatedly, you can orchestrate multiple tool calls into a single composite operation. These workflows can then be referenced by multiple VirtualMCPServer instances, allowing for centralized management of complex tool logic.
  4. What is a VirtualMCPServer

    main

    A VirtualMCPServer is a Custom Resource Definition (CRD) in ToolHive that enables the aggregation of multiple backend MCP (Model Context Protocol) servers into a single, unified virtual endpoint.

    Key capabilities include:

    • Unified authentication: Provides a single authentication point for clients to access multiple backends.
    • Backend discovery: Automatically discovers backend authentication configurations.
    • Tool aggregation: Handles intelligent conflict resolution when multiple backends expose tools with identical names.
    • Composite tools: Allows defining workflows that orchestrate calls across multiple backends.
    • Token caching: Improves performance through efficient token exchange and caching.
  5. What is a Virtual MCP Server (vmcp)?

    main

    A Virtual MCP Server (vmcp) acts as an MCP Gateway. It aggregates tools, resources, and prompts from multiple backend MCP servers into a single unified interface. It is responsible for:

    • Resolving naming conflicts when backends expose duplicate tool names.
    • Executing composite workflows across multiple backends.
    • Handling two-boundary authentication (managing both incoming client authentication and outgoing backend authentication).
  6. Overview of ToolHive Kubernetes Operator Architecture

    main

    The ToolHive operator manages Model Context Protocol (MCP) servers through a layered architecture of Custom Resource Definitions (CRDs). The architecture is divided into several functional layers:

    • Core: Manages the actual running workloads (MCPServer, MCPRemoteProxy, MCPServerEntry).
    • Organization: Groups related servers using MCPGroup.
    • Aggregation: Combines multiple servers into a single endpoint using VirtualMCPServer and VirtualMCPCompositeToolDefinition.
    • Discovery: Uses MCPRegistry to help clients find available servers.
    • Configuration: Provides shared, reusable settings that can be attached to any layer (e.g., MCPToolConfig, MCPExternalAuthConfig, MCPOIDCConfig, MCPTelemetryConfig, MCPWebhookConfig).
    • Auxiliary: Supports infrastructure like EmbeddingServer for semantic search.
  7. Follow CLI naming conventions for commands and flags

    main

    Command Names

    • Use verbs for actions: run, stop, list, remove.
    • Keep names short and memorable.
    • Avoid abbreviations/acronyms unless universally understood.
    • Provide aliases for common commands (e.g., ls for list, rm for remove).

    Flag Names

    • Use lowercase with hyphens: --format, --remote-auth.
    • Use consistent names for common functionality:
      • --all: Show all items (including stopped/hidden).
      • --format: Output format (json/text).
      • --group: Filter/target by group.
      • --debug: Enable debug logging.
    • Use short flags sparingly, only for very frequent options.
    var listCmd = &cobra.Command{
        Use:     "list",
        Aliases: []string{"ls"},
        Short:   "List running MCP servers",
        ...
    }
  8. Security constraints for the Skills System

    main

    The ToolHive skills system implements several security layers to ensure safe execution and installation:

    Archive Extraction Safety

    • Size Limits: Maximum 500MB total decompressed size, 100MB per file, and 1000 files max.
    • Symlink/Hardlink Rejection: Archives containing symlinks or hardlinks are rejected.
    • Path Traversal Prevention: Rejects archives with .. components or absolute paths.
    • Permission Sanitization: Strips setuid/setgid/sticky bits; permissions are capped at 0644.
    • Validation: Performs pre-extraction parent path checks and post-extraction filesystem anomaly scans.

    Path and Supply Chain Protection

    • Dangerous Path Protection: Refuses to remove filesystem roots, home directories, or shallow paths (< 4 components).
    • OCI Naming: The OCI artifact skill name must match the last path component of the OCI repository.
    • Git Security: Git authentication is host-scoped (e.g., GitHub tokens are only sent to github.com). SSRF prevention rejects localhost and private IPs in git references.

    Input Validation

    • Skill Names: Must be 2-64 characters, lowercase alphanumeric plus hyphens, with no consecutive hyphens.
    • Limits: Maximum 64KB for frontmatter and 100 dependencies per skill.
  9. Understand vMCP Multi Round-Trip Requests (MRTR) bridge cells

    main

    vMCP acts as a bridge between different generations of MCP clients and backends. Because vMCP terminates the client connection and re-originates backend calls, the behavior of Multi Round-Trip Requests (MRTR) depends on the combination of client and backend eras.

    Downstream clientBackendMRTR shape
    ModernModernStateless pass-through (vMCP relays capabilities and state verbatim)
    ModernLegacyNo MRTR bridge — deliberate. Calls fail with error codes -32021 (if client lacks capability) or -32603 (if client has capability but it's unsupported). Use the Tasks extension for stateful paths.
    LegacyModernIn-request bridge: vMCP fulfills inputRequests via existing Legacy forwarding seams and retries the backend call within the same request scope.
    LegacyLegacyUnchanged server-initiated forwarding.
  10. Create Composite Tools for multi-step workflows

    main

    Composite tools are new tools defined in vMCP that orchestrate calls to multiple backend tools, allowing complex workflows to be executed without the client knowing the underlying complexity.

    Steps in a composite tool form a Directed Acyclic Graph (DAG). Steps without dependencies execute in parallel, while dependent steps wait for prerequisites.

    Step Types:

    • tool: Executes a backend tool.
    • elicitation: Requests user input via the MCP elicitation protocol.
    • forEach: Iterates over a collection from a previous step, executing an inner tool step per item with bounded parallelism.
  11. Manage Secrets in RunConfig

    main

    Secrets should be referenced by name in the RunConfig rather than being embedded as plaintext. The actual values are resolved at runtime from a provider.

    Supported Providers

    • encrypted: Password-protected local storage.
    • 1password: Integration with 1Password SDK for enterprise vaults.
    • environment: Standard CI/CD environment variables.

    Pattern

    The RunConfig contains the secret name and the target environment variable name. This ensures that serialized RunConfig files do not contain sensitive data and that secret updates do not require RunConfig changes.

  12. Understand the vMCP Multi Round-Trip Requests (MRTR) protocol

    main

    The 2026-07-28 revision of the Model Context Protocol (MCP) replaces server-initiated requests (like sampling/createMessage) with the Multi Round-Trip Requests (MRTR) pattern.

    In this pattern, a server cannot initiate a request to a client. Instead, if a server cannot complete a request (specifically for tools/call, resources/read, or prompts/get), it must respond with a Result where the resultType is "input_required".

    The MRTR Workflow:

    1. Initial Request: The client sends a request (e.g., tools/call).
    2. Input Required Response: The server responds with an InputRequiredResult containing inputRequests (the data needed) and an optional requestState (an opaque string).
    3. Client Fulfillment: The client gathers the required inputs.
    4. Retry: The client sends a new JSON-RPC request with a different id than the original. This retry must include the inputResponses and the echoed requestState from the server.

    Key Constraints:

    • Independence: Each round is a fully independent request; any server replica can handle any round.
    • Capability Gating: Servers MUST NOT send inputRequests for capabilities that the client has not explicitly declared in its clientCapabilities.
    • Security: The requestState is considered attacker-controlled input. Servers must treat it as untrusted and should use cryptographic binding (like HMAC/AEAD) if the state influences authorization or business logic.
    // schema/draft/schema.ts
    export type InputRequest = CreateMessageRequest | ListRootsRequest | ElicitRequest;
    export interface InputRequests { [key: string]: InputRequest; }
    export interface InputRequiredResult extends Result {
      inputRequests?: InputRequests;
      requestState?: string;   // opaque to the client
    }
    export interface InputResponseRequestParams extends RequestParams {
      inputResponses?: InputResponses;
      requestState?: string;
    }