Snowflake Terraform Provider

repository·main·Indexed 20 days ago

https://github.com/snowflakedb/terraform-provider-snowflake

The official Snowflake Terraform Provider allows users to manage Snowflake resources using Terraform HCL. This documentation includes details on the bettertestspoc package for acceptance testing, featuring automated assertion frameworks for Snowflake objects and resources, as well as configuration model builders for resources, datasources, and providers.

Tokens
314.8K
Snippets
728
Records
1.1K
Agent score
70%

What's inside terraform-provider-snowflake

  1. Overview of the Better Tests POC package

    main

    The bettertestspoc package provides a framework of helpers designed to make acceptance tests for the Snowflake Terraform provider quicker, more readable, and more pleasant to implement. It focuses on two main areas: automated assertions and improved configuration modeling.

    Core Packages

    assert package

    This package contains all assertion logic and utilities for building assertions for new objects. Most assertions are currently generated. Supported assertion types include:

    • Snowflake object assertions: Located in the objectassert subpackage.
    • Snowflake object parameters assertions: Located in the objectparametersassert subpackage.
    • Resource assertions: Located in the resourceassert subpackage.
    • Resource parameters assertions: Located in the resourceparametersassert subpackage.
    • Describe / show output assertions: Located in the resourceshowoutputassert subpackage.

    config package

    This package provides model abstractions to improve how configuration is prepared in acceptance tests. It is designed to be more readable and reusable than using hardcoded strings in Config: or using ConfigFile: for files that are not directly reachable from the test body. The following models are available and are already generated:

    • ResourceModel
    • DatasourceModel
    • ProviderModel
  2. Manage Snowflake services with snowflake_service

    main

    The snowflake_service resource is used to manage long-running services in Snowflake (Snowpark Container Services). These services behave like web services that Snowflake manages automatically; for example, if a container stops, Snowflake will restart it to ensure uninterrupted service.

    Important Limitations & Behaviors:

    • State Management: The provider has limited support for managing service state. You can control suspension and resumption via the auto_suspend_secs and auto_resume fields. The provider does not support managing state using ALTER ... SUSPEND or ALTER ... RESUME commands.
    • Deletion: When deleting or recreating a resource, the provider uses DROP SERVICE with the FORCE option to ensure services with block storage volumes are properly handled.
    resource "snowflake_service" "example" {
      database        = snowflake_database.test.name
      schema          = snowflake_schema.test.name
      name            = "SERVICE"
      in_compute_pool = snowflake_compute_pool.test.name
      from_specification {
        stage = snowflake_stage.basic.fully_qualified_name
        file  = "spec.yaml"
      }
    }
  3. Manage streams on views with snowflake_stream_on_view

    main

    The snowflake_stream_on_view resource is used to manage Snowflake streams that monitor views.

    Important Behavior Note: Certain fields such as view, append_only, at, before, show_initial_rows, and stale cannot be modified using an ALTER command in Snowflake. Consequently, changing these fields in your Terraform configuration will trigger a recreation of the resource.

    To ensure access permissions are preserved during this recreation, use the copy_grants = true argument. Without this, the resource is recreated using CREATE OR REPLACE, which may drop existing grants.

    # basic resource
    resource "snowflake_stream_on_view" "stream" {
      name     = "stream"
      schema   = "schema"
      database = "database"
    
      view = snowflake_view.example.fully_qualified_name
    }
    
    # resource with additional fields
    resource "snowflake_stream_on_view" "stream" {
      name     = "stream"
      schema   = "schema"
      database = "database"
    
      copy_grants       = true
      view              = snowflake_view.example.fully_qualified_name
      append_only       = "true"
      show_initial_rows = "true"
    
      at {
        statement = "8e5d0ca9-005e-44e6-b858-a8f5b37c5726"
      }
    
      comment = "A stream."
    }
  4. Manage Iceberg tables from existing metadata files with snowflake_iceberg_table_from_files

    main

    The snowflake_iceberg_table_from_files resource manages an Iceberg table in Snowflake where the metadata is created from an existing Apache Iceberg metadata file located in an external volume.

    ⚠️ Caution: Preview Feature

    This is a preview feature in the provider. It is not guaranteed to be stable, and breaking changes are expected. To use this resource, you must enable the relevant feature in your provider configuration using the preview_features_enabled field.

    Important Limitations

    • Metadata Path Drift: Changes to the metadata_file_path in your external cloud storage are not detected by Terraform. This value is not read back from Snowflake during a refresh. If you update the underlying metadata file externally, Terraform will not see the drift. To apply external changes, you must manually re-create the resource using terraform taint.
    • External Volume Changes: Due to Snowflake limitations, external changes to the external cloud storage type are not detected.
    • Identifier Restrictions: When defining database, name, and schema, avoid using the characters |, ., or " due to technical limitations.
    # Basic - only required fields
    resource "snowflake_iceberg_table_from_files" "basic" {
      database           = "DATABASE"
      schema             = "SCHEMA"
      name               = "TABLE"
      metadata_file_path = "path/to/metadata/v1.metadata.json"
      external_volume    = "my_external_volume"
    }
    
    # Complete - all fields set
    resource "snowflake_iceberg_table_from_files" "complete" {
      database                   = "DATABASE"
      schema                     = "SCHEMA"
      name                       = "TABLE"
      metadata_file_path         = "path/to/metadata/v1.metadata.json"
      external_volume            = "EXTERNAL_VOLUME"
      catalog                    = "CATALOG"
      comment                    = "COMMENT"
      replace_invalid_characters = true
    }
  5. Manage Snowflake tasks with snowflake_task

    main

    The snowflake_task resource is used to manage Snowflake task objects. Tasks allow you to schedule SQL statements or create task graphs (DAGs) for automated execution.

    Important Security and Configuration Warnings

    • Sensitive Values: The config, show_output.config, and show_output.definition fields are not marked as sensitive in the provider. Do not enter personal, sensitive, or regulated data into these fields.
    • Required Fields: Snowflake has complex conditional logic for required fields based on whether a task is serverless, scheduled, or part of a task graph. Always verify your specific use case against the official Snowflake task documentation.
    • Deprecated/Restricted Parameters: Setting AUTOCOMMIT to false or SEARCH_PATH to any value is no longer possible for tasks in Snowflake. It is recommended to avoid configuring these. You may explicitly set autocommit = true to override higher-level hierarchy settings.
    resource "snowflake_task" "task" {
      database  = "database"
      schema    = "schema"
      name      = "task"
      warehouse = "warehouse"
      started   = true
      schedule {
        minutes = 5
      }
      sql_statement = "select 1"
    }
  6. Manage MCP server objects with snowflake_mcp_server

    main

    The snowflake_mcp_server resource is used to manage Model Context Protocol (MCP) server objects in Snowflake.

    Caution: Preview Feature This is a preview feature. Stability is not guaranteed, and breaking changes are expected even without a major version bump. To use this resource, you must enable the relevant feature in your provider configuration using the preview_features_enabled field.

    For detailed information on MCP servers, refer to the Snowflake MCP server documentation.

    resource "snowflake_mcp_server" "example" {
      database = "DB"
      schema   = "SCHEMA"
      name     = "MCP_SERVER"
      specification = "..."
    }
  7. Manage streams on external tables with snowflake_stream_on_external_table

    main

    The snowflake_stream_on_external_table resource is used to manage Snowflake streams that monitor external tables.

    Important Note on Resource Recreation: Certain fields such as external_table, insert_only, at, before, and stale cannot be modified using an ALTER command in Snowflake. Consequently, changing these fields in your Terraform configuration will trigger a recreation of the resource. Even if Terraform marks the change as an update, it will actually perform a replacement. To preserve access permissions during this recreation, set copy_grants = true.

    resource "snowflake_stream_on_external_table" "stream" {
      name     = "stream"
      schema   = "schema"
      database = "database"
    
      external_table = snowflake_external_table.example.fully_qualified_name
    }
  8. Manage Snowflake listings with snowflake_listing

    main

    The snowflake_listing resource is used to manage Snowflake listing objects. A listing allows you to share data or applications with other Snowflake accounts.

    Important Limitations & Behaviors

    • External Changes: The provider does not automatically detect changes made to the manifest file externally (whether inlined or staged). To apply changes made outside of Terraform, you must manually trigger an update, for example, by using terraform taint snowflake_listing.example.
    • Public Listings: This resource is currently not recommended for public listings because the Snowflake review process for public listings does not align well with Terraform's lifecycle. Review requests are closely tied to the publish field.
    • Organization Listings: Support for organization listings is not currently available.
    • External Resources: If your manifest references external resources (like a company logo), you must source your manifest from a stage. References to external resources must be relative to the manifest's location in the stage.
    • Versioning: Versioning only works if the listing sources its manifest from a stage.
    resource "snowflake_listing" "example" {
      name = "MY_LISTING"
      manifest {
        from_string = "title: my title\nsubtitle: my subtitle"
      }
    }
  9. Understand the new resource identifier format

    main

    The provider has updated how resource identifiers are represented:

    1. Single-identifier resources: If a resource can be uniquely identified by one Snowflake identifier, the resource identifier will contain the fully_qualified_name.
    2. Multi-part resources: If a resource requires multiple parts to identify it (e.g., certain grant resources), the resource identifier uses a pipe-separated (|) text format containing all necessary parts.
  10. Enable the HIERARCHY_RENAMES experiment for object renames

    main

    The HIERARCHY_RENAMES experiment allows changing parent identifier fields (like database on snowflake_schema or database/schema on snowflake_table) without forcing resource recreation. The provider detects if a parent was renamed or if the object should be moved (e.g., using ALTER SCHEMA ... RENAME TO ...) and handles it in-place. This prevents data loss and unnecessary recreation of objects.

    Supported resources:

    • snowflake_schema
    • snowflake_table
    provider "snowflake" {
      experimental_features_enabled = ["HIERARCHY_RENAMES"]
    }
    
    # Recommended: Use implicit dependencies
    resource "snowflake_database" "example" {
      name = "my_database"
    }
    
    resource "snowflake_schema" "example" {
      name     = "my_schema"
      database = snowflake_database.example.name
    }
    
    resource "snowflake_table" "example" {
      name     = "my_table"
      database = snowflake_database.example.name
      schema   = snowflake_schema.example.name
    
      column {
        name = "id"
        type = "NUMBER(38,0)"
      }
    }
  11. Identifier character recommendations and limitations

    main

    To ensure reliable parsing and avoid errors, follow these recommendations when naming Snowflake objects (including columns and arguments):

    • Use underscores: Use underscores (_) for word separation instead of special characters.
    • Avoid dots (.): Dots are the primary separator for identifier parts; using them inside an identifier can cause parsing failures.
    • Avoid pipes (|): Pipes are used as separators for complex resource identifiers in the provider.
    • Avoid parentheses (( and )): Avoid these in identifiers for functions, procedures, or external functions to prevent parser splitting issues.
    • Avoid double quotes ("): Do not use double quotes as part of an identifier name.
    • Case Sensitivity: All identifier fields (e.g., name, database, schema) are case-sensitive. If you specify a name in lowercase in Terraform, it must be quoted in Snowflake to match.

    Example of case sensitivity issue in Snowflake SQL: If you have a role named test (lowercase) in Terraform, you must use quotes in SQL:

    show grants to role "test"; -- Correct
    show grants to role test;   -- Incorrect (Snowflake converts unquoted 'test' to uppercase 'TEST')
  12. Inspect Iceberg table details via snowflake_iceberg_tables

    main

    The snowflake_iceberg_tables data source provides three main read-only output collections for each discovered table:

    1. show_output: Contains metadata from the SHOW ICEBERG TABLES command (e.g., base_location, catalog_name, external_volume_name, iceberg_table_format_version, and partition_specs).
    2. describe_output: Contains column-level metadata from the DESCRIBE ICEBERG TABLE command (e.g., name, type, is_nullable, comment, default).
    3. parameters: Contains table-level parameters from the SHOW PARAMETERS FOR ICEBERG TABLE command (e.g., catalog, external_volume, target_file_size).

    Note: To optimize performance and reduce the number of API calls, you can set with_describe and with_parameters to false if you only need the basic show_output information.