Confluent MCP Server

repository·main·Indexed 41 days ago

https://github.com/confluentinc/mcp-confluent

An open-source Model Context Protocol (MCP) server that enables AI assistants to interact with Confluent Cloud, Confluent Platform, and standalone Apache Kafka deployments using natural language. Version 1.6.0 provides over 50 tools covering Kafka, Flink SQL, Schema Registry, Connectors, and Tableflow.

Tokens
28.9K
Snippets
85
Records
123
Agent score
50%

What's inside Confluent MCP Server

  1. Overview of Confluent MCP Server

    main

    The Confluent MCP Server is an open-source Model Context Protocol (MCP) server that allows AI assistants (such as Claude Desktop, Claude Code, Cursor, VS Code, Goose, and Gemini CLI) to interact with Confluent Cloud, Confluent Platform, and standalone Apache Kafka deployments using natural language. It provides over 50 tools covering Kafka, Flink SQL, Schema Registry, Connectors, Tableflow, and more.

    Note on Managed vs. Open Source:

    • Managed MCP Server: Confluent offers a fully managed version for Confluent Cloud customers that requires no local installation and uses existing RBAC permissions.
    • Open Source MCP Server: Use this version if you need support for Confluent Platform/self-managed Kafka or if you wish to customize the toolset. This is a community-supported project.
  2. What telemetry and error reporting covers

    main

    The Confluent MCP Server tracks specific usage events and reports runtime errors to Sentry.

    Tracked Events

    • Tool call completions and failures.
    • Server startup.
    • Confluent Cloud authentication outcomes (success or failure reason).
    • Clicks on the "Copy" button for the agent-skills install command on the OAuth success page.

    Captured Information

    • Tooling: Tool name, execution duration, and success/error status.
    • Authentication: For successful logins, the ccloudUserId (Confluent Cloud resource_id) and ccloudDomain (domain portion of the user's email) are included.
    • Context: Every event includes a random per-process server session ID, server version, transport type, connected MCP client name/version, and OS platform/version/architecture.

    An anonymous user is identified via a machine ID persisted in ~/.mcp-confluent/machine-id.

  3. Configure Direct connections (`type: direct`)

    main

    Direct connections use credentials (like API keys/secrets) defined directly in the YAML configuration. These values can be ${VAR}-interpolated from your shell environment.

    Key characteristics:

    • Service Blocks: You must include at least one service block. Tools that depend on a missing block will be automatically disabled.
    • Read-Only Mode: You can set read_only: true for a connection. This automatically disables all mutation tools (e.g., produce, create, delete) while keeping read-only tools available. This is recommended for staging or production environments.
    • Description: An optional description field provides a label that is returned by the list-configured-connections tool, helping agents distinguish between connections.
    server:
      transports: [stdio]
      log_level: info
    
    connections:
      staging:
        type: direct
        description: "Confluent Cloud Staging via API Keys"
        kafka: { ... }
        schema_registry: { ... }
        confluent_cloud: { ... }
        flink: { ... }
        tableflow: { ... }
        telemetry: { ... }
  4. Understand the three roles of environment variables

    main

    Environment variables serve three distinct purposes in mcp-confluent. Understanding these is critical for correct setup:

    1. YAML Interpolation Source: Used to fill ${VAR} placeholders in your config.yaml. This is the recommended way to manage secrets.
    2. Linked-Library Passthrough: Variables consumed directly by underlying dependencies (e.g., OpenSSL, SASL, Kerberos, or Undici). These must be present in the environment as there are no YAML configuration options for them (e.g., SSL_CERT_FILE, HTTPS_PROXY).
    3. Legacy Configuration Source: In the deprecated mode (starting without -c), environment variables like KAFKA_API_KEY act as the sole source of configuration. This mode is slated for removal.

    You can use the -e <file> CLI flag to load a .env file, which will then supply values for all three roles at startup.

  5. Configure OAuth connections (`type: oauth`)

    main

    OAuth connections allow Confluent Cloud login via PKCE. When a tool requires Cloud access, a browser window opens for authentication. The resulting session is reused for subsequent calls, eliminating the need for manual API key management.

    Key characteristics:

    • No service blocks (like kafka or flink) are defined within the connection block.
    • Resource IDs (such as cluster or env) are passed as arguments to the tools at runtime.
    • Not all tools are currently OAuth-eligible.
    server:
      transports: [stdio]
      log_level: info
    
    connections:
      ccloud-oauth:
        type: oauth
        description: "Confluent Cloud Production"
  6. Filter enabled tools using allow-lists and block-lists

    main

    You can restrict which tools the MCP server exposes by using allow-lists or block-lists.

    Logic Order:

    1. The allow-list is applied first.
    2. The block-list is applied second.

    Usage Patterns:

    • Allow only specific tools: Use --allow-tools with a comma-separated list. All other tools will be disabled.
    • Block specific tools: Use --block-tools with a comma-separated list. All other tools will remain enabled.
    • Use files for lists: If you have many tools, use --allow-tools-file <file> or --block-tools-file <file> (one tool name per line).
    # Allow only specific tools
    npx @confluentinc/mcp-confluent -c config.yaml --allow-tools produce-message,consume-messages
    
    # Block certain tools
    npx @confluentinc/mcp-confluent -c config.yaml --block-tools produce-message,consume-messages
    
    # Use tool lists from files
    npx -y @confluentinc/mcp-confluent -c config.yaml --allow-tools-file allow.txt --block-tools-file block.txt
  7. How tool enablement works based on configuration

    main

    Tools in the Confluent MCP Server are automatically enabled based on the presence of specific service blocks in your configuration.

    • Always available: search-product-docs, get-product-doc-page, explain-disabled-tools, list-configured-connections, config-help, describe-configured-connection.
    • Kafka tools: Enabled if kafka block has bootstrap_servers or type: oauth. REST tools require rest_endpoint + auth.
    • Schema Registry tools: Enabled if schema_registry block is present or type: oauth is used.
    • Flink tools: Enabled if flink block is present. Profiling requires telemetry as well.
    • Tableflow tools: Enabled if tableflow block is present.
    • Confluent Cloud tools: Enabled if confluent_cloud block is present or type: oauth is used.
    • Telemetry tools: Enabled if telemetry block is present.

    Use the --list-tools flag to see the active toolset for your current configuration, or use the explain-disabled-tools tool to diagnose why a specific tool is unavailable.

  8. Use variable interpolation in YAML

    main

    To keep secrets out of version control, you can use ${VAR} or ${VAR:-default} syntax within your config.yaml. The server will substitute these placeholders with values from your shell environment or a loaded .env file.

    Important:

    • If a variable is missing and no default is provided via :-default, the configuration will fail to parse.
    • Plain literals are also supported if you do not require interpolation.
    connections:
      production:
        type: direct
        kafka:
          bootstrap_servers: "broker.confluent.cloud:9092"
          auth:
            type: api_key
            key: "${KAFKA_API_KEY}"
            secret: "${KAFKA_API_SECRET}"
  9. Configure multiple connections in YAML

    main

    A single config.yaml can define multiple named entries under the connections: key. This allows you to manage several environments simultaneously, such as a local Kafka broker and a Confluent Cloud instance. Each tool call is routed to the specific connection identified by its ID.

    Constraints:

    • You can define any number of type: direct connections.
    • You can define at most one type: oauth connection. A second OAuth connection will cause a startup error: Multiple OAuth connections defined in configuration; only one is supported.

    Zero-connection mode: If you define no connections, the server still provides five connection-independent diagnostic and documentation tools:

    • search-product-docs
    • get-product-doc-page
    • explain-disabled-tools
    • list-configured-connections
    • describe-configured-connection
  10. Privacy and data redaction in error reporting

    main

    Error reporting via Sentry is enabled by default in published builds but is inactive in local/source builds. It reports uncaught exceptions, unhandled rejections, and errors thrown by tool handlers (including the tool name and stack trace).

    Security Guarantee: Confluent credentials are never reported. A redaction step automatically scrubs the following before any event is sent:

    • Authorization/Bearer/Basic header values.
    • Confluent API-key/secret values.
    • SASL passwords.
    • Secrets in YAML payloads.

    Note that HTTP bodies and local variables are not collected at all.

  11. Configure HTTP/SSE transport security and API keys

    main

    HTTP and SSE transports require API-key authentication by default.

    1. Generate a key: Run npx @confluentinc/mcp-confluent --generate-key.
    2. Configure the server: Add the key to server.auth.api_key in your config.yaml.
    3. Authenticate clients: Clients must include the key in the cflt-mcp-api-key header.

    To protect against DNS rebinding, the server validates the Host header against server.auth.allowed_hosts (defaults to [localhost, 127.0.0.1]). To disable authentication for local development only, set server.auth.disabled: true or use the --disable-auth flag.

    # Generate key
    npx @confluentinc/mcp-confluent --generate-key
    
    # Example curl request
    curl -H "cflt-mcp-api-key: $MCP_API_KEY" http://localhost:8080/mcp
    server:
      transports: [http]
      auth:
        api_key: "${MCP_API_KEY}"
        allowed_hosts: [localhost, 127.0.0.1, myhost.local]
  12. Prerequisites for running mcp-confluent

    main

    Before installing or running the server, ensure your environment meets these requirements:

    • Node.js: Version 22.19.0 or later (use nvm install 22 && nvm use 22).
    • pnpm: Required if building from source. Install via Homebrew (brew install pnpm) or npm (npm install -g pnpm).
    • Infrastructure: A running Kafka/Schema Registry instance or a Confluent Cloud account.
    • Tableflow Requirements: If using Tableflow tools, you must first configure IAM permissions and provider integrations in Confluent Cloud (e.g., AWS S3 and AWS Glue) following the official Confluent guide.