valkey-go

repository·main·Indexed 20 days ago

https://github.com/valkey-io/valkey-go

A high-performance Golang client for Valkey featuring automatic pipelining and server-assisted client-side caching. It includes a fluent command builder, a mock package for testing based on gomock, and an 'om' package for Generic Object Mapping to map Go structs to Valkey Hashes or JSON documents. Additionally, it provides the 'valkeyaside' package for implementing the Cache-Aside pattern and supports RediSearch queries through OM repositories.

Tokens
80.5K
Snippets
245
Records
356
Agent score
68%

What's inside valkey-go

  1. Use the valkeycompat.Adapter for a go-redis like API

    main

    The valkeycompat.Adapter provides a high-level API that closely mimics the go-redis Cmdable interface. This is useful for developers who prefer a more concise syntax than the standard valkey-go command builder or those migrating existing codebases from go-redis to valkey.

    To migrate, replace your go-redis UniversalClient with a valkeycompat.Adapter initialized with a valkey.Client.

    import (
    	"github.com/valkey-io/valkey-go"
    	"github.com/valkey-io/valkey-go/valkeycompat"
    )
    
    // ... setup client ...
    compat := valkeycompat.NewAdapter(client)
    // Use compat as a go-redis replacement
  2. Implement the Cache-Aside pattern with valkeyaside

    main

    The valkeyaside package provides an implementation of the Cache-Aside pattern enhanced by Valkey Client-Side Caching. This approach addresses common issues like stale cache overrides and cache stampedes.

    Key benefits include:

    • Reduced Network Round Trips: Valkey proactively invalidates the client-side cache.
    • Cache Stampede Prevention: Uses locking (similar to valkeylock) so only the first cache miss triggers a database update, while subsequent callers wait for notifications.
  3. Enable optimistic locking with the version field

    main

    To prevent lost updates in a concurrent environment, add an int64 field to your struct with the valkey:",ver" tag.

    When using this feature:

    1. repo.Save(ctx, entity) will check if the version in Valkey matches the version in your struct.
    2. If they do not match, Save() returns ErrVersionMismatch.
    3. If they match, the save succeeds and the version in Valkey is automatically incremented.

    If you do not include a field with the valkey:",ver" tag, Save() will always succeed without version checking.

    type Example struct {
        Key string `json:"key" valkey:",key"` 
        Ver int64  `json:"ver" valkey:",ver"` // Enables optimistic locking
    }
    
    // If Ver is modified externally, this will fail:
    exp.Ver = 0
    err := repo.Save(ctx, exp)
    if err == om.ErrVersionMismatch {
        // Handle conflict
    }
  4. How Auto Pipelining works

    main

    By default, valkey-go uses Auto Pipelining for all concurrent non-blocking commands (like GET or SET). This automatically reduces round trips and system calls by grouping commands. You can leverage this simply by calling client.Do() from multiple goroutines concurrently.

    func BenchmarkPipelining(b *testing.B, client valkey.Client) {
      b.RunParallel(func(pb *testing.PB) {
        for pb.Next() {
          client.Do(context.Background(), client.B().Get().Key("k").Build()).ToString()
        }
      })
    }
  5. How valkeylock works with Client-Side Caching

    main

    The valkeylock implementation leverages Valkey Client-Side Caching to provide high-performance distributed locking:

    1. Acquisition: It attempts to acquire a majority of keys (e.g., 2 out of 3 if KeyMajority is 2) using SET NX PXAT (or SET NX PX if FallbackSETPX is true).
    2. Success: If a majority is reached within KeyValidity, WithContext returns. The Locker then periodically extends the lock and watches for client-side caching notifications.
    3. Failure/Loss: If the majority is lost (nodes go down or keys are deleted), the Locker receives a notification and immediately cancels the returned ctx.
    4. Waiting: If acquisition fails, the client waits for client-side caching notifications to trigger a retry, rather than polling.
  6. How valkeylimiter works

    main

    The valkeylimiter module implements a Fixed Window Algorithm for distributed rate limiting. It uses Valkey to maintain counters, ensuring consistency across multiple distributed environments.

    Key Mechanisms:

    • Atomicity: It uses Lua scripts executed within Valkey to ensure that checking and updating rate limits are atomic operations, preventing race conditions.
    • Automatic Resets: It leverages Valkey's expiration capabilities to automatically reset rate limits after the specified time window, which ensures efficient memory usage.
    • Client Feedback: The limiter provides ResetAtMs timestamps, allowing your application to inform clients exactly when they can retry requests.
  7. Handle Context Cancellation in Commands

    main

    Methods like client.Do(), client.DoMulti(), client.DoCache(), and client.DoMultiCache() can return early if the provided context deadline is reached.

    Important: Even if the method returns early due to a timeout, the command has likely already been sent to the server.

    To ensure manual context cancellation is respected (especially for blocking requests on dedicated connections), you can use ClientOption.AlwaysPipelining to start pipeline mode in advance.

    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
    defer cancel()
    client.Do(ctx, client.B().Set().Key("key").Value("val").Nx().Build()).Error() == context.DeadlineExceeded
  8. Configure Availability Zone (AZ) Affinity Routing

    main

    For Valkey 8.1+, you can route read requests to replicas in the same availability zone to reduce latency.

    1. Set EnableReplicaAZInfo: true in ClientOption.
    2. If using AWS ElastiCache (Valkey 7.2+), also set AZFromInfo: true.
    3. Provide a ReadNodeSelector function. You can use built-in helpers:
      • valkey.PreferReplicaNodeSelector: Any replica, fallback to primary.
      • valkey.AZAffinityNodeSelector(az): Replica in specific AZ, then any replica, then primary.
      • valkey.AZAffinityReplicasAndPrimaryNodeSelector(az): Replica in specific AZ, then primary in same AZ, then any replica, then primary.

    You can also implement a custom ReadNodeSelector with the signature func(slot uint16, nodes []valkey.NodeInfo) int.

    client, err := valkey.NewClient(valkey.ClientOption{
      InitAddress:         []string{"address.example.com:6379"},
      EnableReplicaAZInfo: true,
      SendToReplicas: func(cmd valkey.Completed) bool {
        return cmd.IsReadOnly() && !cmd.NoReply()
      },
      ReadNodeSelector: valkey.AZAffinityNodeSelector("us-east-1a"),
    })
  9. Set record expiry with the exat tag

    main

    You can automate record expiration by adding a time.Time field to your struct with the valkey:",exat" tag. When repo.Save(ctx, entity) is called, the repository will issue a PEXPIREAT command to set the expiry based on that timestamp.

    Note: If the time.Time field is set to its zero value, the expiry of the record in Valkey remains unchanged.

    type Example struct {
        Key  string    `json:"key" valkey:",key"` 
        ExAt time.Time `json:"exat" valkey:",exat"` 
    }
    
    exp := repo.NewEntity()
    exp.ExAt = time.Now().Add(time.Hour) // Record expires in 1 hour
    repo.Save(ctx, exp)
  10. Enable OpenTelemetry Tracing and Connection Metrics

    main

    To enable OpenTelemetry tracing and connection metrics, use valkeyotel.NewClient instead of the standard client constructor. This provides built-in observability for connection attempts, latency, and client-side caching.

    Built-in Connection Metrics:

    • valkey_dial_attempt: number of dial attempts
    • valkey_dial_success: number of successful dials
    • valkey_dial_conns: number of connections
    • valkey_dial_latency: dial latency in seconds

    Client-side Caching Metrics:

    • valkey_do_cache_miss: number of cache misses on client side
    • valkey_do_cache_hits: number of cache hits on client side

    Client-side Command Metrics:

    • valkey_command_duration_seconds: histogram of command duration
    • valkey_command_errors: number of command errors

    Note: valkeyotel.NewClient is not supported on Go 1.18 and Go 1.19 builds.

    client, err := valkeyotel.NewClient(valkey.ClientOption{InitAddress: []string{"127.0.0.1:6379"}})
    if err != nil {
        panic(err)
    }
    defer client.Close()
  11. Use Generic Object Mapping (OM) with Valkey

    main

    The om package provides Generic Object Mapping to map Go structs to Valkey Hashes or JSON documents. You can create a repository using om.NewHashRepository (for Valkey Hashes) or om.NewJSONRepository (for RedisJSON).

    To use OM, define a struct with specific valkey tags:

    • valkey:",key": Required. Identifies the field used as the unique key (e.g., a ULID).
    • valkey:",ver": Optional. Enables optimistic locking using an int64 field to prevent lost updates.
    • valkey:",exat": Optional. Used on a time.Time field to set record expiry via PEXPIREAT using a Unix timestamp.
    • json:",field": Both repository types use standard JSON tags to determine field names in Valkey.
    type Example struct {
        Key  string    `json:"key" valkey:",key"`   // Required key field
        Ver  int64     `json:"ver" valkey:",ver"`   // Optional for optimistic locking
        ExAt time.Time `json:"exat" valkey:",exat"` // Optional for record expiry
        Str  string    `json:"str"`                // Field name determined by json tag
    }
    
    // Create a Hash repository
    repo := om.NewHashRepository("my_prefix", Example{}, client)
    
    // Create and save an entity
    exp := repo.NewEntity()
    exp.Str = "mystr"
    repo.Save(ctx, exp)
  12. Instantiate a new Valkey Client

    main

    You can create a new Valkey client using valkey.NewClient with a valkey.ClientOption. The library supports several connection modes:

    • Single Node: Provide InitAddress with a slice of addresses.
    • Standalone with Replicas: Use StandaloneOption to provide ReplicaAddress. Use SendToReplicas to define which commands should be routed to replicas (e.g., using cmd.IsReadOnly()).
    • Cluster: Provide multiple addresses in InitAddress. Use ShuffleInit: true to randomize initial connection order.
    • Sentinel: Provide Sentinel addresses in InitAddress and specify the MasterSet in SentinelOption.
    • Unix Socket: Provide the socket path in InitAddress and use DialCtxFn to implement the unix network dialer.
    // Connect to a single valkey node:
    client, err := valkey.NewClient(valkey.ClientOption{
        InitAddress: []string{"127.0.0.1:6379"},
    })
    
    // Connect to a valkey cluster
    client, err := valkey.NewClient(valkey.ClientOption{
        InitAddress: []string{"127.0.0.1:7001", "127.0.0.1:7002", "127.0.0.1:7003"},
        ShuffleInit: true,
    })
    
    // Connect to sentinels
    client, err := valkey.NewClient(valkey.ClientOption{
        InitAddress: []string{"127.0.0.1:26379", "127.0.0.1:26380", "127.0.0.1:26381"},
        Sentinel: valkey.SentinelOption{
            MasterSet: "my_master",
        },
    })