DOCSEARCHbyAlgolia
Submit Docs

dbt-score

repository·master·Indexed 42 days ago

https://github.com/picnicsupermarket/dbt-score

A linting and scoring tool for dbt metadata that ensures project quality by defining generic and custom rules. It analyzes dbt entities including Models, Sources, Snapshots, Exposures, Seeds, and Macros using the manifest.json file. The tool supports custom rule creation via the @rule decorator or Rule class, rule filtering, and provides machine-readable JSON output for CI/CD integration.

Tokens
12.8K
Snippets
50
Records
66
Agent score
53%

What's inside dbt-score

  1. What is dbt-score and what can it lint?

    master

    dbt-score is a linter for dbt metadata. It allows you to lint and score metadata associated with dbt entities to enforce or encourage good practices.

    Currently, dbt-score can lint the following dbt entities:

    • Models
    • Sources
    • Snapshots
    • Exposures
    • Seeds
    • Macros

  2. Filter rules using @rule_filter or RuleFilter

    master

    Filters allow you to ignore specific dbt entities for certain rules. A filter returns True if the item should be evaluated, and False if it should be ignored.

    Using @rule_filter (Decorator)

    Use the @rule_filter decorator on a function. The argument type annotation dictates the target entity.

    from dbt_score import Model, RuleFilter, rule_filter

@rule_filter
def only_schema_x(model: Model) -> bool:
    """Only applies a rule to schema X."""
    return model.schema.lower() == 'x'

    Using RuleFilter (Class)

    Inherit from RuleFilter and implement the evaluate method.

    class SkipSchemaY(RuleFilter):
   description = "Applies a rule to every schema but Y."
   def evaluate(self, model: Model) -> bool:
      return model.schema.lower() != 'y'

    Applying filters to a rule

    You can attach filters to a rule using the rule_filters argument in the @rule decorator.

    @rule(rule_filters={only_schema_x()})
def models_in_x_follow_naming_standard(model: Model) -> RuleViolation | None:
    # ... implementation
    from dbt_score import Model, rule, RuleViolation
from my_project import only_schema_x

@rule(rule_filters={only_schema_x()})
def models_in_x_follow_naming_standard(model: Model) -> RuleViolation | None:
    """Models in schema X must follow the naming standard."""
    if some_regex_fails(model.name):
        return RuleViolation("Invalid model name.")

  3. How dbt-score discovers rules via namespaces

    master

    By default, dbt-score searches for rules in two specific Python namespaces:

    1. dbt_score.rules: Contains the core rules packaged with dbt-score.
    2. dbt_score_rules: An implicit namespace package used for custom rules, whether they are provided by 3rd party packages or implemented by you.

    You can configure dbt-score to search for rules in additional namespaces via the main configuration settings.

  5. Implement custom rules for multiple projects or distribution

    master

    To share rules across multiple dbt projects in an organization or distribute them publicly, package your rules in a Python wheel.

    The wheel must expose modules within a top-level dbt_score_rules namespace (or another namespace configured in your settings).

    To prevent naming conflicts, use module names that reflect your organization or specific project. For example, a project named dbtviz should place its rules in dbt_score_rules.dbtviz.

  6. Set up the dbt-score development environment

    master

    To set up a local development environment for dbt-score, ensure you have Python 3.10 or higher, pre-commit, and uv installed. After cloning the repository, run the following commands from the project root to install git hooks and all necessary dependency groups (including development dependencies):

    pre-commit install
uv sync --all-groups

  7. Implement custom rules for a single dbt project

    master

    To write custom rules that apply only to a specific project, bundle them directly within your dbt project directory. dbt-score automatically injects the current working directory into the Python path, making a dbt_score_rules directory discoverable without extra configuration.

    Recommended project structure:

    my-dbt-project/
├─ dbt_score_rules/
│  ├─ my_project_rules.py
├─ dbt_project.yml
├─ models/
├─ ...

    Ensure that the rules are importable within your Python environment.

  9. Run tests and check coverage

    master

    Testing is performed using pytest, coverage, and tox.

    Quick local testing:

    • Run all tests: uv run pytest
    • Run tests with coverage reporting: uv run coverage run -m pytest

    Testing specific combinations with tox:

    • Test a specific Python and dbt version (e.g., Python 3.13 and dbt 1.1.1): uv run tox -e py313-dbt111
    • Test all Python versions against a single dbt version: uv run tox -f dbt111
    • Test a single Python version against all supported dbt versions: uv run tox -f py312
    uv run pytest
uv run coverage run -m pytest
uv run tox -e py313-dbt111
uv run tox -f dbt111
uv run tox -f py312

  10. Run linting and type checking

    master

    The project uses ruff for linting/formatting, mypy for type checking, and tox for linting environments. Use these commands to check or fix code quality:

    • Check linting: uv run ruff check .
    • Auto-fix linting: uv run ruff check --fix
    • Type check: uv run mypy .
    • Run linting via tox: uv run tox -e lint
    uv run ruff check .
uv run ruff check --fix
uv run mypy .
uv run tox -e lint