DOCSEARCHbyAlgolia
Submit Docs

dbt-mcp

repository·main·Indexed 20 days ago

https://github.com/dbt-labs/dbt-mcp

A Model Context Protocol (MCP) server that provides AI agents with tools to interact with dbt resources, including dbt Core, dbt Fusion, and dbt Platform. It enables agents to execute SQL, query the Semantic Layer, inspect project lineage, manage jobs, and generate dbt code. The repository includes integration examples for AG2 (formerly AutoGen), Vercel AI SDK, CrewAI, and AWS Agentcore.

Tokens
31K
Snippets
93
Records
129
Agent score
69%

What's inside dbt-mcp

  1. Overview of dbt MCP Server Tools

    main

    The dbt MCP server provides a suite of tools categorized by functional area to allow AI agents to interact with dbt Core, dbt Fusion, and dbt Platform.

    Key tool categories include:

    • SQL: Execute and generate SQL on dbt Platform.
    • Semantic Layer: Query metrics, dimensions, and entities.
    • Discovery: Inspect project metadata (models, macros, lineage, etc.).
    • dbt CLI: Execute standard dbt commands (run, test, build, etc.).
    • Admin API: Manage dbt Platform jobs and projects.
    • dbt Codegen: Automate YAML and SQL boilerplate generation.
    • dbt LSP: Advanced SQL compilation and column-level lineage.
    • Product Docs: Search and fetch content from official dbt documentation.
    • Metadata: Inspect the MCP server version and branch.

  2. Overview of dbt Agentcore Tools

    main

    The system uses a multi-agent architecture with three specialized tools:

    1. dbt Compile Tool (dbt_compile.py): Performs local dbt project compilation using dbt compile --log-format json. It parses JSON logs to provide error analysis and recommendations.
    2. dbt Model Analyzer Tool (dbt_model_analyzer.py): Analyzes model structure, dependencies, data quality patterns, test coverage, and adherence to dbt best practices.
    3. dbt MCP Server Tool (dbt_mcp.py): Connects to a remote dbt MCP server via a streamable HTTP client. It supports dbt Cloud authentication, dynamic tool listing, and intelligent query routing.

  3. How the AG2 multi-agent system works

    main

    The multi-agent implementation separates concerns between two specialized agents to ensure safe and efficient dbt operations:

    1. analyst_agent: A read-only agent. It uses Discovery, Semantic Layer, and SQL tools to investigate the dbt project and formulate recommendations.
    2. executor_agent: An action-oriented agent. It uses dbt CLI, Admin API, and Codegen tools to execute recommended actions. To prevent unauthorized warehouse modifications, it runs in human_input_mode="ALWAYS", acting as an approval gate.

    Agent Handoff Logic

    AG2 manages transitions between these agents using a four-priority handoff system evaluated on every turn:

    1. Context conditions: Deterministic logic (e.g., blocking the executor until a specific condition is met) that does not require an LLM call.
    2. LLM conditions: The analyst_agent decides to hand off when it determines an action is warranted.
    3. Tool-based handoffs: A tool can trigger a direct handoff to the next agent via a ReplyResult.
    4. After-work fallback: If no other conditions are met, both agents return control to the user.

  4. Traverse the lineage graph

    main

    When processing a LineageGraph, use the edges list to navigate relationships:

    • Finding Upstream Dependencies (Parents): Look for edges where the target is your node; the source of those edges is the parent.
    • Finding Downstream Dependents (Children): Look for edges where the source is your node; the target of those edges is the child.

    Common Use Cases

    • Impact Analysis: To see what might break if a model changes, follow edges downstream (where source is the target node).
    • Dependency Tracking: To see what a model depends on, follow edges upstream (where target is the target node).
    • Finding Tests: Filter the nodes list for entries where resource_type == "Test" to identify associated tests.

  5. Run the OpenAI Agent with a Local MCP Server

    main

    To use the OpenAI Agent integration with a local MCP server, follow these steps:

    1. Configure your environment variables as described in the main project README.
    2. Ensure that the MCPServerStdio environment variable points to your local MCP server configuration.
    3. Set the OPENAI_API_KEY environment variable with your valid OpenAI API key.
    4. Execute the agent using uv.
    # Ensure OPENAI_API_KEY is set
export OPENAI_API_KEY='your-api-key'
# Ensure MCPServerStdio is set to your local server path
export MCPServerStdio='path/to/your/server'

uv run main.py

  6. Remove a tool (Phase B) instructions

    main

    Once tool usage is approximately zero over a 30-day window, perform the removal in a dedicated PR:

    1. Delete tool definitions: Remove from discovery/tools.py, tools_multiproject.py, and the DISCOVERY_TOOLS / MULTIPROJECT_DISCOVERY_TOOLS entries.
    2. Delete metadata/mappings: Remove from tools/tool_names.py, tools/toolsets.py, and tools/readme_mappings.py.
    3. Delete prompts: Remove the prompt file from prompts/discovery/.
    4. Clean up deprecation helpers: Remove the deprecated_description / deprecation_meta call sites added during the deprecation phase.
    5. Clean up client code: Delete unused client code (verify with grep first).
    6. Update tests: Update round-trip tests in test_tool_names.py and test_toolsets.py to remove the tool from the expected set.
    7. Changelog: changie new --kind "Breaking Change" --body "Removed <tool>."
    8. Generate assets: task docs:generate and task contract:generate (commit the updated snapshot).
    9. Verify: task check + task test:unit.
    10. Release: Follow the standard release and rollout process (Major version bump).

  7. Install AG2 for dbt-mcp integration

    main

    To use dbt-mcp with the AG2 (formerly AutoGen) multi-agent framework, install the ag2 package with openai and mcp extras. Ensure you have followed the project's root README instructions to configure the necessary dbt-mcp environment variables first.

    pip install "ag2[openai,mcp]>=0.11.0"

  8. Verify the dbt AWS Agentcore setup

    main

    Before running the full application, use the provided test scripts to verify your environment and connectivity.

    Quick MCP connectivity test (verifies environment variables and lists available MCP tools):

    python dbt_data_scientist/quick_mcp_test.py

    Comprehensive test suite (tests all tools, agent initialization, and integration):

    python dbt_data_scientist/test_all_tools.py
    # Quick MCP test
python dbt_data_scientist/quick_mcp_test.py

# Full test suite
python dbt_data_scientist/test_all_tools.py

  9. Trigger a dbt job run

    main

    Use the trigger_job_run tool to start a new execution for a specific dbt job. You can provide optional parameter overrides to modify the execution context, such as changing the Git branch, overriding the target schema, or providing custom execution steps.

    Parameters

    • job_id (required): The unique identifier of the dbt job to run.
    • cause (optional): A description of why the job was triggered.
    • git_branch (optional): The Git branch to use for this run.
    • schema_override (optional): A schema name to override the default execution schema.
    • steps_override (optional): A list of custom dbt commands to execute instead of the default job steps (e.g., ["dbt run --select my_model"]).

    Returns

    A run object containing:

    • Run ID and status
    • Job and environment information
    • Git branch and SHA being used
    • Trigger information and cause
    • Execution queue position
    {
  "job_id": 456,
  "cause": "Manual trigger for testing"
}

  10. Run the LangGraph Agent Example

    main

    This example demonstrates how to build a conversational agent using LangGraph integrated with a remote dbt MCP server. To run the example, you must provide an Anthropic API key via environment variables and use uv to execute the script.

    export ANTHROPIC_API_KEY='your-api-key-here'
uv run main.py

  11. Configure the Google ADK Agent for dbt MCP

    main

    To use the Google Agent Development Kit (ADK) with the remote dbt MCP server, you must configure several environment variables for both Google Generative AI and dbt connectivity.

    Google ADK Configuration:

    • GOOGLE_GENAI_API_KEY: The API key for the Google model (or any other model supported by Google ADK).
    • ADK_MODEL: The specific model to use. Defaults to gemini-2.0-flash.

    dbt Configuration:

    • DBT_TOKEN: Your dbt authentication token.
    • DBT_PROD_ENV_ID: The ID of your dbt production environment.
    • DBT_HOST: The dbt host URL (defaults to cloud.getdbt.com if not provided).
    • DBT_PROJECT_DIR: Required if you are using dbt Core instead of dbt Cloud.
    export GOOGLE_GENAI_API_KEY='your_key'
export ADK_MODEL='gemini-2.0-flash'
export DBT_TOKEN='your_token'
export DBT_PROD_ENV_ID='your_env_id'
export DBT_HOST='cloud.getdbt.com'

  12. Configure dbt MCP via the UI

    main
    The dbt MCP UI provides a graphical interface to simplify the configuration of the dbt MCP server. For specific command-line instructions on how to run, build, or manage the UI, refer to the package.json scripts or the Taskfile.yml definitions in the repository.