dbt_project_evaluator

The dbt_project_evaluator package identifies areas in a dbt project that deviate from dbt Labs' best practices. It evaluates your project across six key dimensions:

1. Modeling: DAG modeling best practices.
2. Testing: Model testing best practices.
3. Documentation: Model documentation best practices.
4. Structure: File structure and naming conventions.
5. Performance: Materialization strategies for performance.
6. Governance: Model governance features.

Beyond running tests, the package generates a model named int_all_dag_relationships, which provides a tabular representation of your DAG that can be queried via SQL in your data warehouse.

The int_all_dag_relationships model is a core mart provided by the dbt_project_evaluator package. It contains a comprehensive list of all dbt nodes (models, exposures, sources, metrics, seeds, and snapshots) along with their dependencies (both direct and indirect) and the specific paths between them. You can build custom models, snapshots, or dashboards on top of this table to analyze your project's structure.

-- Example: Querying the relationships model
select * from {{ ref('int_all_dag_relationships') }}

The fct_root_models metric identifies models with zero direct parents, meaning they cannot be traced back to a declared source or model in the dbt project.

Why it matters: This usually means the model uses raw table references (e.g., FROM raw_database.raw_table) instead of {{ source() }} or {{ ref() }}. This breaks dbt's lineage tracking, which can lead to incorrect execution timing or circular references.

Exceptions: Manually defined reference tables that are self-contained, such as a dim_calendar generated via {{ dbt_utils.date_spine() }}.

How to remediate: Map the raw table references in the FROM clause to their corresponding dbt models or sources. Replace them with {{ ref() }} for models or {{ source() }} for raw data sources.

The fct_source_fanout metric flags instances where a single source is the direct parent of multiple resources in the DAG.

Why it matters: Best practices suggest each source should be referenced by exactly one staging model. This staging model acts as a buffer to perform light transformations (renaming, recasting) to ensure consistency and DRY (Don't Repeat Yourself) code. Referencing sources directly makes the project harder to maintain if raw data formats change.

Exceptions: Heavily nested or NoSQL data sources where a single raw table must be broken into multiple base models.

How to remediate: Create a single staging model that references the source and performs necessary cleaning. Refactor all other models that were referencing the source directly to point to this new staging model instead.

The fct_staging_dependent_on_marts_or_intermediate metric flags staging models that depend on intermediate or marts models (based on your project's naming conventions and folder paths).

Why it matters: Staging models should only select from source nodes. If a staging model depends on a downstream model, it is likely misnamed or misconfigured.

How to remediate: Either rename the staging model to a more appropriate prefix (e.g., int_ or fct_) or refactor its lineage to point directly to the appropriate {{ source() }}.

The fct_unused_sources metric identifies sources that have zero children (no models reference them).

Why it matters: This represents cruft in the project, such as sources defined in YML that were never implemented, or sources belonging to deprecated models that were not removed from the YML files.

How to remediate: Locate the corresponding sources.yml file and remove the unused table name and its associated metadata (descriptions, etc.).

sources:
  - name: some_source
    database: raw
    tables:
      - name: table_1
      - name: table_2
      - name: table_3
      - name: table_4  # <-- remove this line

The package evaluates your dbt project against best practices in six categories: Modeling, Testing, Documentation, Structure, Performance, and Governance.

It operates in three stages:

1. Graph Parsing: It parses your dbt graph object and writes it to your warehouse as a series of models (located in models/marts/core). This includes the int_all_dag_relationships model, which provides a tabular view of your DAG.
2. Misalignment Modeling: It creates models representing specific types of misalignments (rule violations).
3. Testing: It runs tests against those models to alert you to misalignments via test warnings.

To troubleshoot a warning, you can query the associated model to find specific instances of the issue or customize the package to exclude specific rules.

The fct_documentation_coverage model calculates the percentage of enabled models in your project that have a configured description.

By default, this model will raise a warn error during dbt build or dbt test if the documentation_coverage_pct is less than 100%. You can customize this behavior by overriding the documentation_coverage_target variable to set a different threshold.

To remediate low coverage, ensure every model has at least a model-level description in its .yml file or via a {{ doc() }} block.

The fct_staging_dependent_on_staging metric flags parent/child relationships where both models reside in the staging layer.

Why it matters: This suggests a naming error or a lineage error. If one staging model depends on another, the child model is likely actually an intermediate or marts model.

How to remediate: Either change the model type of the child (e.g., move it to an intermediate directory and rename it to int_...) or change its lineage to reference a {{ source() }} instead.

To execute integration tests locally using an Athena target, you must first configure the required environment variables and update your profiles.yml to use them. Once configured, run your dbt commands using the --target athena flag.

# 1. Set environment variables
export ATHENA_S3_STAGING_DIR=your_staging_dir
export ATHENA_S3_DATA_DIR=your_data_dir
export ATHENA_REGION=your_region
export ATHENA_SCHEMA=your_schema
export ATHENA_WORKGROUP=your_workgroup

# 2. Run dbt commands with the athena target
dbt test --target athena