ScyllaDB gocql Driver

repository·master·Indexed 42 days ago

https://github.com/scylladb/gocql

A shard-aware fork of the apache/cassandra-gocql-driver optimized for ScyllaDB. It reduces latency by avoiding inter-shard data movement through extensions to TokenAwareHostPolicy and support for shard-aware ports. Features include LZ4 and Snappy compression, AddressTranslator for cloud IP mapping, and a generic, thread-safe EventBus package for event processing in Go.

Tokens
48.7K
Snippets
174
Records
249
Agent score
53%

What's inside gocql

  1. Overview of ScyllaDB gocql driver

    master
    ScyllaDB gocql is a shard-aware fork of the Apache Cassandra GoCQL driver. It maintains compatibility with the github.com/gocql/gocql package API while introducing ScyllaDB-specific optimizations, most notably shard-aware routing and support for shard-aware ports.
  2. Use prepared statements and bind markers

    master

    The driver automatically handles the preparation and caching of DML statements. To use them, use bind markers (?) for values in your CQL string.

    Token-Aware Routing: To enable automatic token-aware routing, every component of the partition key must be provided as a bound value via a bind marker.

    Concurrency Warning: Do not reuse a Query object across multiple goroutines. Always create a new Query from the shared Session for each concurrent operation.

    query := session.Query(
    	"SELECT payload FROM events WHERE account_id = ? AND event_id = ?",
    	accountID,
    	eventID,
    )
  3. Handle empty intermediate pages in range or filtering queries

    master

    When performing range queries or queries using ALLOW FILTERING, the server may return an empty intermediate page. Do not assume an empty page means the entire result set has been exhausted.

    To correctly determine if more data is available, use Iter.Scan, Iter.LastPage, or check the length of the returned page state instead of treating an empty page as completion.

  4. Avoid DowngradingConsistencyRetryPolicy unless necessary

    master

    The DowngradingConsistencyRetryPolicy may return results with a lower consistency level than originally requested. Use this only if reduced consistency is an explicit requirement of your application.

    Behavioral Notes:

    • It does not downgrade queries already identified as an LWT.
    • It can downgrade conditional statements that were not identified as LWTs before execution (e.g., after a preparation or binding failure).
  5. How SimpleRetryPolicy handles Lightweight Transactions (LWT)

    master

    SimpleRetryPolicy retries normal queries on a different host. However, if the driver identifies a query as a Lightweight Transaction (LWT) before the first attempt, it will retry on the same host to minimize Paxos contention.

    Note that conditional statements (using Session.Bind or explicit routing keys) might only be identified as LWTs after execution begins. If you are implementing a custom policy that requires specific LWT behavior, you must implement both the RetryPolicy and LWTRetryPolicy interfaces.

  6. Understand shard-aware connection routing

    master

    When ScyllaDB advertises a supported, nonzero shard configuration, the driver automatically routes connections to specific shards and opens exactly one connection per shard. In this mode, the ClusterConfig.NumConns setting is ignored.

    If server-side shard-aware drivers are disabled or if valid sharding metadata is unavailable, the driver reverts to its standard connection pool behavior and honors the NumConns configuration.

  7. How EventBus works

    master

    EventBus is a generic, thread-safe event processing package for Go based on channels. It allows multiple subscribers to receive events from a single input channel.

    Key behaviors:

    • Non-Blocking Distribution: The bus uses non-blocking sends to subscriber channels. If a subscriber's buffer is full, the event is dropped for that specific subscriber only, preventing slow subscribers from blocking the entire bus or other subscribers.
    • Generic Support: It uses Go generics to work with any type T.
    • Lifecycle: When EventBus.Stop() is called, all subscriber channels are closed. When a subscriber is stopped via Subscriber.Stop() or EventBus.Unsubscribe(), only that subscriber's channel is closed.
    • Thread Safety: All public methods are safe for concurrent use by multiple goroutines.
  8. Enable token-aware routing by binding partition keys

    master

    For TokenAwareHostPolicy to work, the driver needs metadata for every component of the partition key. To ensure the driver can identify the correct partition, you must bind all partition-key values as parameters in your query instead of embedding them directly into the CQL string.

    // Routable: both partition-key components are bound values.
    session.Query(
    	"SELECT value FROM events WHERE account_id = ? AND bucket = ?",
    	accountID,
    	bucket,
    )
  9. Implement custom retry policies safely

    master

    When implementing a custom RetryPolicy, the driver does not provide an automatic idempotence guard. Your implementation of GetRetryType must manually inspect errors to decide whether a retry is safe.

    To make safe decisions, inspect the following on the *gocql.QueryError:

    • PotentiallyExecuted(): Returns true for ambiguous client-side failures (like request timeouts or transport errors after the request was sent).
    • IsIdempotent(): Indicates if the query is marked as idempotent.
    • Server errors: Such as *gocql.RequestErrWriteTimeout.

    Additionally, you can set ClusterConfig.DefaultIdempotence to change the default for all queries, though per-query marking is preferred.

  10. How to manage gocql Sessions

    master

    A Session is the primary object for interacting with a ScyllaDB cluster. It handles cluster topology discovery, connection pool maintenance, and request routing.

    Best Practice: Create exactly one Session during your application startup and share it across all goroutines. Do not create a session per request, as this causes excessive connection overhead and redundant topology discovery work.

    cluster := gocql.NewCluster(
    	"192.0.2.10:9042",
    	"192.0.2.11:9042",
    	"192.0.2.12:9042",
    )
    cluster.Keyspace = "application"
    
    session, err := cluster.CreateSession()
    if err != nil {
    	return err
    }
    defer session.Close()