DOCSEARCHbyAlgolia
Submit Docs

dbt Agent Skills

repository·main·Indexed 20 days ago

https://github.com/dbt-labs/dbt-agent-skills

A collection of specialized instructions and scripts that enable AI agents to perform dbt-related tasks, including analytics engineering, semantic layer management, and migrations. Includes the skill-eval CLI for scaffolding and executing LLM skill evaluations, and support for installation via Claude Code, Vercel Skills CLI, and Tessl.

Tokens
78.6K
Snippets
187
Records
310
Agent score
70%

What's inside dbt-agent-skills

  1. Use the 'Answering Natural Language Questions with dbt' skill

    main

    This skill is designed to answer business questions about analytics, metrics, KPIs, or data (e.g., "What were total sales last month?") by writing and executing SQL queries.

    When to use:

    • Answering user questions about business metrics or data.
    • Exploring data via the Semantic Layer or ad-hoc SQL.

    When NOT to use:

    • Validating dbt model logic during development.
    • Testing dbt models or semantic layer definitions.
    • Building or modifying dbt models.
    • Running dbt run, dbt test, or dbt build workflows.

  2. Use the 'using-dbt-for-analytics-engineering' skill

    main

    The using-dbt-for-analytics-engineering skill is designed for building and modifying dbt models, writing SQL transformations using ref() and source(), creating tests, and validating results with dbt show.

    Use this skill for:

    • Building new dbt models, sources, or tests.
    • Modifying existing model logic or configurations.
    • Refactoring dbt project structures.
    • Creating analytics pipelines or data transformations.
    • Working with warehouse data that needs modeling.

    Do NOT use this skill for:

    • Querying the semantic layer (use answering-natural-language-questions-with-dbt instead).
    • Making breaking changes to models with consumers (e.g., renaming, removing, or retyping a column). For breaking changes, use the working-with-dbt-mesh skill to implement model versioning.

  3. What is dbt State and how does it work?

    main

    dbt State is a server-backed reuse mechanism that decides whether a node should be skipped, cloned, or built before execution.

    Unlike the state:modified selector (which compares file hashes) or --state deferral (which only handles missing upstream nodes), dbt State:

    • Automatically manages state on a server.
    • Parses SQL into a syntax tree to compare semantic hashes (ignoring whitespace/comments).
    • Considers upstream data freshness to decide if a node is stale.
    • Rebuilds descendants only if they actually depend on a change.

    Reuse Decision Logic:

    1. Skip: The object exists in the target schema, its semantic hash is unchanged, and no parent has fresher data beyond lag_tolerance.
    2. Clone: A matching object (same hash, fresh data) exists in another schema (e.g., production or a teammate's dev schema). It uses zero-copy cloning or CTAS to reuse the data. Test results are also reused.
    3. Build: No valid reuse option is found; the node builds normally and auto-defers unselected upstream nodes.

  4. Compare manifest.json lineage retrieval vs MCP tools

    main

    Use the manifest.json fallback method when you cannot access MCP lineage tools.

    Featuremanifest.json Fallback
    Requirementmanifest.json must exist (requires dbt parse)
    ConnectivityWorks offline; no MCP server required
    Data CompletenessContains complete lineage and all metadata
    PerformanceCan be slow/memory-intensive for large projects (>100MB)
    AccuracyReflects the last parse; does not include uncommitted changes

  5. Design Metrics using Type Progression

    main

    When building metrics, start with the simplest form and advance as complexity requirements grow. Every metric must include the following properties: name, description, label, and type.

    Metric Types

    1. Simple: A single aggregation with optional filters. This is the recommended starting point.
    2. Ratio: A calculation where a numerator is divided by a denominator.
    3. Derived: Calculations that combine multiple existing metrics.
    4. Cumulative: Running totals or windowed aggregations.

  6. What are dbt unit tests and when to use them

    main

    dbt unit tests validate SQL modeling logic on static inputs before materializing in production. If a unit test fails, dbt will not materialize the model.

    When to use

    • To prevent regressions when changing model logic.
    • To verify bug fixes.
    • For models with complex logic (Regex, Date math, Window functions, complex case when statements, or complex joins).
    • For custom logic processing input data (similar to a function).
    • For high-criticality models (public, contracted, or upstream of an exposure).
    • To handle edge cases not yet seen in actual data.
    • Prior to significant refactoring.

    When not to use

    • For built-in warehouse functions (e.g., min()) that are already extensively tested by the provider.

  7. Choose the correct model access level

    main

    Use the following logic to determine the appropriate access level for a model:

    1. Is it referenced cross-project?
      • Yes $\rightarrow$ access: public (contracts recommended).
      • No $\rightarrow$ Proceed to step 2.
    2. Is it referenced outside its group?
      • Yes $\rightarrow$ access: protected (default).
      • No $\rightarrow$ Proceed to step 3.
    3. Is it internal to a small team?
      • Yes $\rightarrow$ access: private.
      • No $\rightarrow$ access: protected (default).

    Best practice: Default new models to private and widen access only when necessary.

  8. Strategize data test placement in the dbt pipeline

    main

    To avoid duplicating tests for pass-through columns and to maximize signal, apply different testing strategies at each layer of your dbt project:

    • Staging Layer: Focus on Data Hygiene. Catch formatting issues, completeness (unexpected nulls), and granularity (duplicates).
    • Intermediate Layer: Focus on Grain and Joins. Test when the grain of the table changes or when joins introduce new risks (e.g., composite keys).
    • Marts Layer: Focus on Business Logic. Protect end-user facing data by testing business expectations and new calculated fields.

  9. Configure SCD Type II dimensions

    main

    For slowly changing dimension (SCD Type II) tables, use the natural entity type and define validity_params on your time dimensions to specify start and end markers.

    Constraints: SCD Type II semantic models cannot contain simple metrics.

    Key configuration:

    • primary_entity: Set at the model level.
    • entity: {type: natural}: Used for the primary key.
    • dimension: {validity_params: {is_start: true}}: Used on the start time column.
    • dimension: {validity_params: {is_end: true}}: Used on the end time column.
    models:
  - name: sales_person_tiers
    semantic_model:
      enabled: true
    agg_time_dimension: tier_start
    primary_entity: sales_person
    columns:
      - name: start_date
        granularity: day
        dimension:
          type: time
          name: tier_start
          validity_params:
            is_start: true
      - name: end_date
        granularity: day
        dimension:
          type: time
          name: tier_end
          validity_params:
            is_end: true
      - name: sales_person_id
        entity:
          type: natural
          name: sales_person

  10. Understand dbt State billing (DATTs)

    main

    dbt State is metered in DATTs (daily active target tables), not by the number of models built.

    • Target Table: A database object managed by your project (seeds, snapshots, models, and each distinct test). For example, a model dim_customers with two tests (not_null and unique) counts as 3 target tables.
    • When a DATT is consumed: A target table becomes a DATT when dbt State performs at least one skip, clone, or test reuse on it in a single day (UTC). A full build is not a reuse.
    • Views: Views are never billed as DATTs, even if reused or cloned. However, tests attached to a view are billed normally.

  11. Handle cross-platform data and configuration differences

    main

    Be aware of these nuances when moving between different data platforms:

    • Platform-specific config keys: Keys like snowflake_warehouse or cluster_by will not cause errors on the source platform. They only surface as errors when compiling against the target platform. Do not remove them prematurely.
    • Dataset variance: Even standard datasets (like TPCH) may have minor schema or data differences (column names, types, row counts) across platforms. Always verify the source data schema on both platforms before assuming 1:1 equivalence during migration testing.

  12. Understand the Skill Evaluation directory structure

    main

    A typical evaluation project follows this structure:

    • scenarios/: Contains test scenarios. Each scenario folder includes:
      • scenario.md: Description and grading criteria.
      • prompt.txt: The user message to send.
      • skill-sets.yaml: Skill combinations and tool permissions.
      • context/: Files required by the LLM (copied to a temp environment).
      • .env: Environment variables for setup commands and MCP servers.
    • runs/: Timestamped output from executions (gitignored). Each run contains:
      • output.md: Full conversation text.
      • metadata.yaml: Metrics (success, tokens, cost, tools used).
      • changes/: Files modified or created during the run.
      • transcript/: HTML files for browser-based viewing.
    • reports/: Generated comparison reports.
    • src/skill_eval/: The CLI source code.