go-redis

The DIGEST command (available in Redis 8.4+) returns a 64-bit xxh3 hash of a key's value. This allows for efficient optimistic locking and change detection without transferring the entire value between the server and the client.

Core Use Cases

• Optimistic Locking: Update a value only if its current digest matches your expected digest (ensuring no one else changed it).
• Change Detection: Update a value only if its current digest differs from your expected digest.
• Conditional Deletion: Delete a key only if its current digest matches a specific value.

Advantages over WATCH/MULTI/EXEC

• Simpler: Uses single commands instead of complex transaction blocks.
• Faster: Reduces network round trips and avoids transaction overhead.
• Client-side friendly: You can calculate the expected digest locally if you already know the value, avoiding an extra DIGEST call to Redis.

import "github.com/redis/go-redis/v9/helper"

// For strings
digest := helper.DigestString("myvalue")

// For binary data
digest := helper.DigestBytes([]byte{0x01, 0x02, 0x03})

Maintenance Notifications provide zero-downtime connection handoffs during Redis cluster maintenance (like Redis Enterprise operations).

Core Mechanism

1. Push Notifications: The client listens for RESP3 push notifications (e.g., MOVING, MIGRATING, FAILING_OVER).
2. Event-Driven Handoff: When a notification is received, an asynchronous worker pool handles the transition to new connections without blocking client operations.
3. Timeout Relaxation: During maintenance, the client automatically relaxes read/write timeouts (RelaxedTimeout) to prevent false failures caused by increased latency during the handoff.
4. Circuit Breaker: To prevent overwhelming failing endpoints, a circuit breaker tracks failures per endpoint and enters a 'half-open' state to test recovery gradually.

Supported Notification Types

• Standalone Client: Supports MOVING, MIGRATING, MIGRATED, FAILING_OVER, and FAILED_OVER.
• Cluster Client: Supports SMIGRATING and SMIGRATED for hitless slot migrations.

Architecture

• Non-Blocking: Uses a worker pool and queue-based architecture to ensure client operations continue during handoffs.
• Thread-Safe: Employs lock-free atomic operations and sync.Map for high-performance state tracking.
• Auto-Scaling: If MaxWorkers or HandoffQueueSize are set to 0, the client automatically calculates optimal sizes based on the connection pool size.

The HIMPORT command family (available in Redis 8.10+) allows for fast hash ingestion using hinted hash templates. This process involves two steps:

1. HIMPORT PREPARE: Registers an ordered list of field names (a fieldset) within a connection's session.
2. HIMPORT SET: Creates hashes by sending only the values, which are paired positionally with the prepared fields. This allows the server to store hashes using a memory-efficient template encoding where shared field names are stored only once.

Client-side Management

go-redis manages fieldsets automatically for pooled callers using the following typed methods:

• HImportPrepare: Registers a fieldset.
• HImportSet: Performs the ingestion using the prepared fieldset.
• HImportDiscard: Discards a specific fieldset.
• HImportDiscardAll: Discards all fieldsets.

go-redis remembers fieldsets client-side and replays the PREPARE command lazily (at most once per connection session) on whichever pooled connection executes an HImportSet. If a connection loses its session state (due to RESET or a reconnect), the client transparently re-prepares the fieldset.

Warning: These guarantees apply only to the typed methods listed above. If you use generic command interfaces like Do or raw Cmd values, the client-side registry is bypassed. In those cases, PREPARE commands will not be replayed on other connections, and no such fieldset errors may occur. For low-level HIMPORT usage, use a dedicated connection via client.Conn().

// Conceptual usage of the typed API
err := client.HImportPrepare(ctx, fieldset).Err()
err = client.HImportSet(ctx, key, values).Err()

Autopipelining automatically batches concurrent commands from multiple goroutines into Redis pipelines. This is intended for high-throughput/high-load scenarios.

Two modes of operation:

1. AutoPipeline() (Blocking): A drop-in replacement for a normal client. Each command call blocks until it executes and returns its own value/error. Per-goroutine ordering is preserved.
2. AsyncAutoPipeline() (Deferred): Highest throughput. Command calls return immediately (returning a command object), and you read the results later. This keeps pipelines deep.

Important Caveats:

• Contexts: A command's context is NOT honored once it is queued; batches execute on the autopipeliner's own context. Use a plain client if you need per-command deadlines.
• Non-idempotent commands: On a dropped connection, a batch is retried as a whole. Non-idempotent commands might execute twice.
• Bypassed commands: Blocking commands (BLPOP, WAIT), SHUTDOWN, MONITOR, and Do bypass batching.

// Blocking face: drop-in for a normal client, batched under the hood.
ap, err := rdb.AutoPipeline()
if err != nil {
    log.Fatal(err)
}
defer ap.Close()

var wg sync.WaitGroup
for i := 0; i < 1000; i++ {
    wg.Add(1)
    go func(i int) {
        defer wg.Done()
        key := fmt.Sprintf("key:%d", i)
        if err := ap.Set(ctx, key, i, 0).Err(); err != nil { // blocks until executed
            log.Printf("set %s: %v", key, err)
        }
    }(i)
}
wg.Wait()

// Async face: for maximum throughput
ctx := context.Background()
ap, err := rdb.AsyncAutoPipeline()
if err != nil {
    log.Fatal(err)
}
defer ap.Close()

cmds := make([]*redis.StatusCmd, 0, 200)
for i := 0; i < 200; i++ {
    cmds = append(cmds, ap.Set(ctx, fmt.Sprintf("key:%d", i), i, 0)) // returns immediately
}
for _, cmd := range cmds {
    if err := cmd.Err(); err != nil { // blocks until executed
        log.Printf("set: %v", err)
    }
}