Go Snowflake Driver

repository·master·Indexed 18 days ago

https://github.com/snowflakedb/gosnowflake

A Go implementation of the Snowflake database driver supporting the standard database/sql package. Version 2 requires Go 1.24+ and supports various authentication methods including OAuth2, JWT keypair, and SSO. It provides advanced features such as Apache Arrow batch retrieval via the arrowbatches package, ArrowStreamLoader for chunked data downloads, and integration with Go's log/slog for client logging.

Tokens
30.4K
Snippets
97
Records
125
Agent score
62%

What's inside gosnowflake

  1. Configure Chunk Download Workers via Session Parameter

    master

    In v2, the global MaxChunkDownloadWorkers variable has been removed. Instead, configure the number of download workers using the Snowflake session parameter CLIENT_PREFETCH_THREADS. The default is 4.

    ALTER SESSION SET CLIENT_PREFETCH_THREADS = 10
  2. Migrate from v1 to v2

    master

    Version 2.0.0 introduced breaking changes. Follow these steps to migrate:

    1. Update Import Paths: Change imports from github.com/snowflakedb/gosnowflake to github.com/snowflakedb/gosnowflake/v2.
    2. Update Arrow Batches: The Arrow batches API has moved to a separate package: github.com/snowflakedb/gosnowflake/v2/arrowbatches. This allows applications to avoid the Arrow compute dependency unless needed.
    3. Update Configuration: Several fields in gosnowflake.Config have been renamed or removed (e.g., InsecureMode is now DisableOCSPChecks).
    4. Update File Transfers: WithFileStream has been split into WithFilePutStream and WithFileGetStream.
    5. Update Context Functions: WithMultiStatement no longer returns an error, and Array now returns an error for unsupported types.
    6. Update Environment Variables: The typo in GOSNOWFLAKE_SKIP_REGISTERATION is fixed to GOSNOWFLAKE_SKIP_REGISTRATION.
  3. Install the Go Snowflake Driver v2

    master

    To install the Go Snowflake Driver v2, ensure you have Go 1.24 or higher installed. Initialize your Go module and then fetch the driver using go get.

    Prerequisites:

    • Go 1.24 or higher
    • Supported OS: 64-bit Linux, Mac, or Windows

    Note: This driver does not currently support GCP regional endpoints.

    go mod init example.com/snowflake
    go get -u github.com/snowflakedb/gosnowflake/v2
  4. Configure OAuth2 Authentication Flows

    master

    The driver supports two primary OAuth2 flows:

    1. Authorization Code Flow (AuthTypeOAuthAuthorizationCode): A browser-based flow typically used for interactive user login.
    2. Client Credentials Flow (AuthTypeOAuthClientCredentials): A non-interactive flow used for machine-to-machine authentication.

    When using these flows, the driver can manage token refreshing. If a token expires, the driver attempts to use a refresh token to obtain a new access token before failing the authentication request.

  5. Understand Snowflake query status codes

    master

    While many internal status constants are deprecated, the driver uses them to interpret the state of a query. Common states include:

    • RUNNING: The query is currently executing.
    • SUCCESS: The query completed successfully.
    • FAILED_WITH_ERROR: The query failed due to a specific error.
    • QUEUED: The query is waiting in the queue.
    • BLOCKED: The statement is waiting on a lock on a resource held by another statement.
    • RESUMING_WAREHOUSE: The query is waiting for a warehouse to resume.
  6. Identify file transfer queries

    master
    The driver can automatically detect if a query is a file transfer operation (specifically PUT or GET commands) using internal regex matching. This allows the driver to switch to the specialized snowflakeFileTransferAgent to handle progress tracking and metadata reporting.
  7. Retrieve QueryID from errors in SnowflakeStmt

    master
    When executing a query via SnowflakeStmt, if an error occurs, the driver attempts to capture the Snowflake QueryID from the error. If the error can be cast to a *SnowflakeError, the GetQueryID() method will return the QueryID contained within that error, allowing you to identify which specific query failed in Snowflake.
  8. Configure OCSP Fail-Open Mode

    master

    The gosnowflake driver supports two OCSP (Online Certificate Status Protocol) fail modes to handle situations where the OCSP responder is unreachable:

    1. Fail-Open (OCSPFailOpenTrue): If the OCSP check fails (e.g., network error, timeout), the driver assumes the certificate is valid and proceeds with the connection. This is the default behavior.
    2. Fail-Closed (OCSPFailOpenFalse): If the OCSP check fails, the driver treats the certificate as potentially revoked and terminates the connection.

    Note: The OCSPFailOpenMode type is currently deprecated and will be moved to Config/DSN in future releases. Use the constants OCSPFailOpenTrue and OCSPFailOpenFalse to set this value.

    // Example of the intended usage (conceptually, as it moves to Config/DSN)
    // mode := gosnowflake.OCSPFailOpenFalse
  9. Use JWT for Keypair Authentication

    master

    To use JWT (JSON Web Token) authentication, you must provide a PrivateKey in your driver configuration. The driver will automatically generate a JWT with the following claims:

    • iss: {account_name}.{user_name}.SHA256:{base64_encoded_public_key_hash}
    • sub: {account_name}.{user_name}
    • iat: Issued at time (UTC)
    • nbf: Not before (fixed to 2015-10-10)
    • exp: Expiration time based on your JWTExpireTimeout configuration.

    Ensure the PrivateKey is valid and the Account and User fields are correctly populated in your Config object.

  10. Understand the results of file transfer operations

    master

    When performing file transfers (PUT/GET) in Snowflake, the driver returns a result set containing metadata about the operation. The structure of this result set depends on whether you performed an upload or a download.

    Upload (PUT) Results

    An upload operation returns a JSON result set with the following columns:

    • source: The source file name.
    • target: The destination file name in the Snowflake stage.
    • source_size: The size of the source file.
    • target_size: The size of the target file in the stage.
    • source_compression: The compression type used for the source.
    • target_compression: The compression type used for the target.
    • status: The status of the transfer.
    • message: Any error or status messages.

    Download (GET) Results

    A download operation returns a JSON result set with the following columns:

    • file: The name of the downloaded file.
    • size: The size of the file.
    • status: The status of the transfer.
    • message: Any error or status messages.

    If an error occurs during the transfer, the driver returns a SnowflakeError with either ErrFailedToUploadToStage or ErrFailedToDownloadFromStage depending on the operation type.

  11. Bypass driver registration with GOSNOWFLAKE_SKIP_REGISTRATION

    master

    By default, the driver registers itself with sql.Register("snowflake", &SnowflakeDriver{}) during init().

    If you need to call multiple versions of the driver within a single client, you can set the environment variable GOSNOWFLAKE_SKIP_REGISTRATION to prevent the driver from registering automatically. This avoids panics caused by registering the same driver name multiple times.

    Note: If you use sql.Open(), you must not use this environment variable, as sql.Open requires the driver to be registered to map the name "snowflake" to the driver type.

    # Set this to avoid registration panics when using multiple driver versions
    export GOSNOWFLAKE_SKIP_REGISTRATION=true
  12. Configure Snowflake Driver via Config struct

    master

    The gosnowflake.Config struct is used to manage connection settings. Note the following changes in v2:

    • ServerSessionKeepAlive: Replaces KeepSessionAlive.
    • DisableOCSPChecks: Replaces InsecureMode.
    • Transporter: Use this field to provide a custom transport (replaces the global SnowflakeTransport).
    • TLSConfigName: Use this to reference a custom TLS configuration registered via RegisterTLSConfig.
    import "crypto/tls"
    
    // Custom Transport
    config := &gosnowflake.Config{
        Transporter: yourCustomTransport,
    }
    
    // Custom TLS
    tlsConfig := &tls.Config{
        // ...
    }
    _ = gosnowflake.RegisterTLSConfig("custom", tlsConfig)
    config.TLSConfigName = "custom"