DOCSEARCHbyAlgolia
Submit Docs

Turntable Documentation

website·Indexed 29 days ago

https://turntable-so.github.io/docs

Documentation for the Turntable VSCode extension, featuring guides on installation, requirements for DBT projects, and tools for generating and viewing column-level lineage and documentation.

Tokens
3.2K
Snippets
8
Records
33
Agent score
75%

What's inside Turntable

  1. Understand Column-Level Lineage (CLL) in Turntable

    Column-level lineage (CLL) provides a detailed visualization of the connections between individual columns across tables in a data warehouse, rather than just table-to-table relationships. It is used to accelerate debugging by showing how a column is calculated without manual code review and to facilitate knowledge sharing within data teams.

  3. Edit model and column documentation via the extension pane

    To edit documentation, modify the model and column description fields in the extension pane and press Save. The extension automatically updates the corresponding YAML file or creates a new one if it doesn't exist. After saving, a notification appears in the bottom right; clicking View Changes opens a diff view of the updated YAML file.

  4. Verify system and software requirements for Turntable installation

    Before installing Turntable, ensure your environment meets the following minimum requirements:

    • Platform: Mac (ARM or Intel), Windows (x86), or Linux (x86 or ARM).
    • Visual Studio Code: Version 1.7.8 or greater.
    • Build Tools: XCode command line tools (Mac), Visual C++ build tools (Windows), or gcc (Linux).
    • Python: Version 3.9 or above.
    • Python Packages: dbt-core and your specific dbt adapter must be installed.
    • dbt Project Health: The project must be able to successfully execute dbt parse, dbt compile, and dbt docs generate --no compile.

  5. Open the Turntable lineage view in VS Code

    To open the lineage view, open a dbt model in your project and click the "Turntable" pane in the VS Code panel. The table-level lineage will load first, followed by column-level lineage a few seconds later. Note: Column-level lineage requires specific pre-requisites to be met before it will function.

  7. Generate documentation using the Autofill feature

    Use the 🪄 Autofill button to automatically generate first-draft column descriptions using AI. The process involves:

    1. Clicking the 🪄 Autofill button.
    2. Selecting the target model and the specific column descriptions to update (fields without existing descriptions are selected by default).
    3. Clicking OK to trigger the AI generation.
    4. Reviewing the generated content and making manual adjustments if necessary.
    5. Clicking Save to commit the changes to the YAML files.

  8. Refresh the lineage view

    Click the 🔄 button at the top right of the pane to refresh the lineage view and return it to the default state.

    Caveats:

    • Refreshing recompiles the model but does not automatically account for upstream or downstream schema changes.
    • Turntable uses the catalog.json file for schema information. You must materialize changes (e.g., by running dbt run) for catalog changes to take effect in the view.

  9. Manage Turntable telemetry and AI data sharing in VSCode

    Users can opt out of telemetry and AI data processing (sending model metadata and code to OpenAI and Anthropic) via the VSCode settings UI. To prevent any data from leaving the local machine, both telemetry and AI options must be disabled.
    1. Press Ctrl + Shift + P
2. Type 'settings' to open the VSCode settings UI
3. To disable telemetry: Search for 'turntable telemetry' and uncheck the box
4. To disable AI data sharing: Search for 'turntable ai' and uncheck the box

  10. Configure Python interpreter and dbt packages in VS Code

    To ensure Turntable uses the correct Python environment:

    1. Select Interpreter: Press Control + Shift + P, type Python: Select Interpreter, and choose the environment with Python 3.9+ (or 3.8+ as per remedy notes). If you cannot find the path, run dbt debug in the terminal and check the python_path output.
    2. Install Adapter: Press Control + Shift + P, type Python: Create Terminal, and install your specific dbt adapter using pip.
    pip install dbt-{adapter}

  11. Prepare requirements for Turntable onboarding

    Before configuring the extension, gather the following project details to ensure a successful onboarding process:

    • Python Interpreter Path: Supported versions are 3.9, 3.10, and 3.11.
    • dbt Version: Supported versions are 1.3, 1.4, 1.5, and 1.6 (Note: dbt 1.3 is incompatible with Python 3.11).
    • dbt Adapter: Supported adapters include Snowflake, Bigquery, Databricks, Redshift, and Postgres.
    • profiles.yml Location: The default location (e.g., ~/.dbt or project directory) or any absolute folder path.
    • Environmental Variables: Any project-specific variables (must be specified individually; .env files are not yet supported).

  12. Address Column-Level Lineage limitations and requirements

    Turntable's CLL approach uses sqlglot to parse dbt code into an abstract syntax tree. To ensure accurate lineage, users should follow these requirements and mitigations:

    • Catalog Dependency: Turntable relies on catalog.json. Ensure you have credentials to run dbt docs generate and that models (and their ancestors) are materialized via dbt run after code updates.
    • SQL Parsing: If SQL cannot be parsed, only table-level lineage is shown. To mitigate this, qualify columns (e.g., table.id instead of id) when using multiple joins.
    • Function References: Table references that are functions (e.g., UNNEST clauses) may result in incomplete lineage. Use SELECT statements where possible.
    • Supported Warehouses: Currently, only BigQuery and Snowflake are supported.