DOCSEARCHbyAlgolia
Submit Docs

YDB Go SDK

repository·master·Indexed 41 days ago

https://github.com/ydb-platform/ydb-go-sdk

A pure Go native client and database/sql driver for YDB, a distributed SQL database. It provides high-level clients for Query, Topic, and Coordination services, and supports various authentication methods including IAM, service account keys, OAuth 2.0 token exchange, and metadata credentials. The SDK includes a database/sql driver for standard SQL patterns and supports integration with ORMs like GORM and Xorm.

Tokens
34.7K
Snippets
114
Records
154
Agent score
51%

What's inside YDB Go SDK

  1. Use the experimental metrics package for monitoring

    master
    The metrics package is an experimental feature in ydb-go-sdk that provides adopter interfaces. These interfaces allow you to integrate the SDK's internal system and common metrics into your own monitoring systems (such as Prometheus, OpenTelemetry, or other observability platforms).

  2. Understand YDB authentication methods in Go

    master

    The ydb-go-sdk supports several authentication mechanisms to connect to YDB. By default, the SDK attempts to connect using environment variables unless specific credentials are provided.

    Available authentication examples include:

    • access_token_credentials: Using an access token.
    • anonymous_credentials: Connecting without authentication (anonymous access).
    • metadata_credentials: Using instance metadata (common in cloud environments).
    • oauth2_token_exchange_credentials: Using OAuth 2.0 token exchange.
    • service_account_credentials: Using a service account key file.
    • static_credentials: Using static credentials.
    • environ: Using environment variables to configure authentication.

  3. Enable query enrichment (bindings) in YDB

    master

    YDB queries often require explicit DECLARE statements for parameters and PRAGMA TablePathPrefix for table scoping. To avoid writing this boilerplate manually, you can enable Query Enrichment (also known as bindings). This allows you to write simple SQL (like SELECT ?, ?) and have the driver automatically expand it into valid YQL.

    Supported Enrichments

    • TablePathPrefix: Automatically adds a PRAGMA TablePathPrefix to scope table names.
    • Declare: Automatically generates DECLARE statements for parameters.
    • Positional: Enables the use of ? positional arguments.
    • Numeric: Enables numeric parameter binding.

    How to enable

    1. Connection String: Add go_auto_bind to your DSN with comma-separated options.
    2. Connector Options: Use query.TablePathPrefix, query.Declare, etc., when initializing the connector via ydb.MustConnector.
    // Using Connection String
db, _ := sql.Open("ydb", "grpc://localhost:2136/local?go_auto_bind=table_path_prefix(/local/path/to/my/folder),declare,positional")

// Using Connector Options
nativeDriver := ydb.MustOpen(ctx, "grpc://localhost:2136/local")
db := sql.OpenDB(ydb.MustConnector(nativeDriver,
    query.TablePathPrefix("/local/path/to/my/folder"),
    query.Declare(),
    query.Positional(),
))

  4. How to work with YDB decimal types in Go

    master

    The ydb/decimal package provides tools for handling YDB's decimal types.

    Key Concepts:

    • Storage: Decimal types are stored as int128 values in YDB.
    • Representation: In the ydb package, they are represented as 16-byte arrays ([16]byte). In the ydb/decimal package, they are represented as *math/big.Int.
    • Scaling: Returned big.Int values are scaled. When performing math operations, you must account for the scaling factor. The default scale is 9 (a factor of $10^9$).

    To perform correct arithmetic, you must scale your operands to match the decimal's scale before calling methods like Add.

    import (
	"ydb/decimal"
	"math/big"
)

// Default scale is 9.
var scaleFactor = big.NewInt(10000000000) 

func main() {
	x := decimal.FromInt128([16]byte{...})

	// Incorrect: adding an unscaled integer
	x.Add(x, big.NewInt(42)) 

	// Correct: adding a scaled integer
	x.Add(x, scale(42)) 
}

func scale(n int64) *big.Int {
	x := big.NewInt(n)
	return x.Mul(x, scaleFactor)
}

  5. Use supported read-only isolation levels in v3

    master

    When migrating to ydb-go-sdk/v3, note that the available isolation levels for read-only interactive transactions have been restricted.

    • In v2: sql.LevelReadCommitted and sql.LevelReadUncommitted were supported (implemented via fake transactions using OnlineReadOnly control).
    • In v3: Only sql.LevelSnapshot is allowed for read-only interactive transactions.

    Currently, sql.LevelSnapshot is implemented using a fake transaction with OnlineReadOnly control on each request. This is a transitional implementation until YDB clusters fully deploy native snapshot isolation, at which point the fake transaction layer will be replaced by true read-only interactive transactions.

  6. Understand the `ydb.RunWithRetry` and `ydb.Try` lifecycle

    master

    The SDK uses a nested span structure to manage retries:

    • ydb.RunWithRetry: The outer span representing the entire operation, including all retry attempts.
    • ydb.Try: An inner span created for every individual attempt. The first attempt's span contains no extra tags.

    Retry Behavior

    • Backoff: Retry attempts (those preceded by a sleep) include the ydb.retry.backoff_ms attribute, indicating the wait time before the attempt.
    • Errors:
      • On a non-retryable error, both the current ydb.Try and the parent ydb.RunWithRetry are marked as failed using error.type and db.response.status_code.
      • On retry exhaustion, all ydb.Try spans and the ydb.RunWithRetry span carry the terminal error.
    • Cancellation: On context cancellation (context.Canceled or context.DeadlineExceeded), the spans carry an error.type set to the dynamic Go type of the cancelling error.

  7. How retry tracing is implemented

    master

    The SDK propagates the driver-level trace.Retry into internal retry loops (e.g., in query.Client.Do and query.Client.DoTx). This ensures that ydb.RunWithRetry and ydb.Try spans are visible for query-service operations, not just database/sql retries.

    The ydb.Try span includes the ydb.retry.backoff_ms attribute, which is populated by the trace.Retry.OnRetryAttempt callback, representing the sleep duration waited before that specific attempt.

  8. Handle truncated results in v3

    master

    YDB has a default limit of 1000 rows. In v3, session.Execute wraps truncated results as a non-retryable error, while session.StreamExecuteScanQuery wraps them as a retryable error. This removes the need to manually check res.Truncated() in most cases.

    If you need to change this behavior, you can use the ydb.WithIgnoreTruncated option.

  9. Understand the YDB-Go-SDK versioning policy

    master

    The ydb-go-sdk follows SemVer 2.0.0 with specific exceptions for experimental, deprecated, and internal features. Understanding these markers helps you manage stability and technical debt in your codebase.

    Feature Classifications

    • Experimental (// Experimental): New features that may be unstable. We reserve the right to modify or remove these features at any time without a major version bump. Early adopters are encouraged to report bugs.
    • Deprecated (// Deprecated): Features marked for future removal. These will remain available for at least six months after the mark is added. You can use linters like staticcheck or go-critic to detect these in your code.
    • Internals (// Internals): Public APIs that are tied to the SDK's internal workings. These are never marked as stable and can change at any time without a major version bump.
    • testutil: The testutil package is considered internal and can change at any time without a major version bump.

  10. Understand session pooling in YDB `database/sql`

    master

    It is important to distinguish between the two types of session pooling used:

    1. Native Driver Session Pool: Managed internally by ydb-go-sdk/v3 (e.g., via db.Table().Do()). Configured via ydb.WithSessionPoolSizeLimit().
    2. database/sql Session Pool: Managed by the standard Go sql.DB implementation. This is configured using standard methods: db.SetMaxOpenConns and db.SetMaxIdleConns.

    The YDB driver automatically transforms internal YDB error codes into sql.driver.ErrBadConn and other retryable/non-retryable types. While the driver handles many errors automatically, complex transaction failures may require the user to wrap logic in retry helpers like retry.Do or retry.DoTx.

  11. Initialize the YDB `database/sql` driver using a DSN

    master

    You can initialize the driver using a standard Data Source Name (DSN) or connection string. This requires importing the driver with a blank identifier.

    DSN Parameters:

    • token: Access token required for requests.
    • Static credentials: Use the format grpcs://root:password@localhost:2135/local.
    • query_mode=scripting: Redefines the default DML query mode.
    import (
    "database/sql"
    _ "github.com/ydb-platform/ydb-go-sdk/v3"
)

func main() {
    db, err := sql.Open("ydb", "grpc://localhost:2136/local")
    if err != nil {
        // handle error
    }
    defer db.Close()
}

  12. Migrate `sql.Connector` initialization from v2 to v3

    master

    In ydb-go-sdk/v3, the process for initializing a database/sql driver via a connector has changed. Unlike v2, which used a single-return ydbsql.Connector that initialized the driver lazily, v3 requires you to first open a native YDB driver using ydb.Open. This native driver is then used to create the connector via ydb.Connector. This change allows for explicit error handling during the initial connection phase and avoids the rare errors associated with lazy initialization.

    nativeDriver, err := ydb.Open(context.TODO(), "grpc://localhost:2136/local")
if err != nil {
    // handle error
}
defer nativeDriver.Close(context.TODO())

connector, err := ydb.Connector(nativeDriver)
if err != nil {
    // handle error
}

db := sql.OpenDB(connector)
defer db.Close()