DOCSEARCHbyAlgolia
Submit Docs

Titan Core Documentation

repository·main·Indexed 19 days ago

https://github.com/titan-systems/titan

A declarative infrastructure-as-code (IaC) tool for Snowflake designed to replace manual SQL scripts and tools like Terraform. Titan Core allows users to provision and secure Snowflake resources—including users, roles, schemas, databases, and warehouses—via a Python API, a CLI using YAML configurations, or a dedicated GitHub Action. It features dynamic role switching, no state file dependency, and support for variable injection and scoped management at the database or schema level.

Tokens
46.6K
Snippets
149
Records
212
Agent score
66%

What's inside Titan Core

  1. Overview of Titan Core

    main
    Titan Core is a declarative infrastructure-as-code (IaC) tool specifically designed for Snowflake. It allows you to provision, deploy, and secure Snowflake resources (such as users, roles, schemas, databases, integrations, pipes, stages, functions, and stored procedures) using simple, repeatable configurations instead of manual SQL scripts. It is designed as a replacement for tools like Terraform, Schemachange, or Permifrost, offering faster execution and a git-based workflow.

  2. Titan Core Documentation Overview

    main

    Titan Core is a Snowflake infrastructure-as-code tool. The documentation is organized into several key areas to help you manage Snowflake resources:

    • Getting Started: Initial setup and first steps.
    • Working With Resources: Detailed guidance on managing Snowflake objects.
    • Blueprint: Information on using Blueprints to group and manage resource sets.
    • GitHub Action: Instructions for integrating Titan into CI/CD pipelines.
    • Resource Reference: A comprehensive list of supported Snowflake resources (e.g., Database, Warehouse, Table, Role, Storage Integrations, etc.).

  3. Configure ExternalAccessIntegration

    main

    Use ExternalAccessIntegration to enable code within Snowflake functions and stored procedures to utilize secrets and establish connections with external networks. This resource defines which NetworkRules and authentication secrets are accessible to the integration.

    To configure this in Titan, you can either instantiate a Python object or define it in a YAML configuration file.

    external_access_integration = ExternalAccessIntegration(
    name="some_external_access_integration",
    allowed_network_rules=["rule1", "rule2"],
    enabled=True
)

  5. Compare Titan Core with other Snowflake IaC tools

    main

    Titan Core is a declarative, Python-based Snowflake infrastructure-as-code tool. It differs from other tools in several key ways:

    • vs Terraform: Unlike the Snowflake provider for Terraform which is limited to 1 role per provider, Titan Core supports dynamic role switching, automatically detecting and switching to the required role for a change. It also has no state file dependency, providing more accurate plans and eliminating stale state issues.
    • vs Schemachange: While Schemachange is an imperative migration tool requiring manual SQL scripts, Titan Core is declarative. You define the desired state, and Titan handles the implementation details.
    • vs Permifrost: Titan Core is designed for high performance, running in seconds even for complex environments, whereas Permifrost can be significantly slower.
    • vs SnowDDL: SnowDDL uses a specific 3-tier role hierarchy model; Titan Core is a more general-purpose declarative tool.

  7. Define a Snowflake Secret

    main

    A Secret defines a set of sensitive data used for authentication or other purposes within Snowflake. You can define secrets using either Python objects or YAML configuration. The name and type fields are required.

    secret = Secret(
    name="some_secret",
    type="OAUTH2",
    api_authentication="some_security_integration",
    oauth_scopes=["scope1", "scope2"],
    oauth_refresh_token="some_refresh_token",
    oauth_refresh_token_expiry_time="some_expiry_time",
    username="some_username",
    password="some_password",
    secret_string="some_secret_string",
    comment="some_comment",
    owner="SYSADMIN",
)

  8. Use the Grant resource to manage Snowflake privileges

    main

    The Grant resource represents the assignment of privileges on a specific Snowflake resource to a role. You can define grants for global account privileges, warehouse privileges, schema privileges, or table privileges using either Python or YAML.

    Key Fields

    • priv (string, required): The specific privilege to grant (e.g., 'SELECT', 'INSERT', 'CREATE TABLE', 'OPERATE').
    • on (string or [Resource], required): The target resource. This can be a string (like 'ACCOUNT') or a specific resource object (like Warehouse(name="foo")).
    • to (string or [Role], required): The role receiving the privileges.
    • grant_option (bool): If true, the grantee can grant these privileges to other roles. Defaults to False.
    • owner (string or [Role]): The role that owns the grant. Defaults to 'SYSADMIN'.
    # Example: Granting SELECT on a table to a role
grant = Grant(priv="SELECT", on_table="sometable", to="somerole")

  9. Manage GCS storage integrations with GCSStorageIntegration

    main

    Use the GCSStorageIntegration resource to manage the integration of Google Cloud Storage (GCS) as an external stage in Snowflake. This allows you to define which GCS locations are accessible for data storage and which are explicitly blocked.

    gcs_storage_integration = GCSStorageIntegration(
    name="some_gcs_storage_integration",
    enabled=True,
    storage_allowed_locations=['gcs://bucket/path/'],
    storage_blocked_locations=['gcs://bucket/blocked_path/']
)

  10. Use variables (vars) in Titan configurations

    main

    Titan supports dynamic configurations using variables (vars). You can define variables in YAML or Python and inject values at runtime.

    YAML Syntax

    Use double curly braces {{ var.name }} for Jinja-style templating. You can define expected variables and their defaults using the top-level vars: key.

    vars:
  - name: color
    type: string
  - name: fruit
    type: string
    default: apple

databases:
  - name: "db_{{ var.color }}_{{ var.fruit }}"

    Python Syntax

    You can use the titan.var module or Jinja-style strings:

    from titan import var
from titan.resources import Database

# Using the var module
db1 = Database(name=var.db1_name)

# Using Jinja-style strings
db2 = Database(name="db_{{ var.db2_name }}")

    Injecting Variable Values

    1. CLI Flag: Pass a JSON string via --vars. titan plan --config titan.yml --vars '{"fruit": "banana"}'
    2. Environment Variables: Use variables starting with TITAN_VAR_ in uppercase. export TITAN_VAR_FRUIT="peach" titan plan --config titan.yml
    3. Python Blueprint: Pass a dictionary to the vars parameter in the Blueprint constructor.
    # Example YAML with vars
vars:
  - name: color
    type: string
  - name: fruit
    type: string
    default: apple

databases:
  - name: "db_{{ var.color }}_{{ var.fruit }}"

  11. Establish relationships between resources

    main

    You can link resources together to define relationships (e.g., assigning a role to a user) using two primary methods:

    1. Pass by Instance: Pass the actual resource object directly to another resource's parameter. This is the clearest and most direct method.
    2. Pass by Name: Pass the resource's name as a string. This is useful for serialization or specific configuration requirements.