Lettuce Java Redis Client

repository·main·Indexed 26 days ago

https://github.com/redis/lettuce

A scalable, thread-safe Java Redis client built on Netty. Lettuce supports synchronous, asynchronous, and reactive programming models (via Project Reactor), as well as advanced Redis features including Sentinel, Cluster, and Pub/Sub. It provides granular control over connection behavior through ClientOptions and resource management via ClientResources.

Tokens
62.6K
Snippets
134
Records
239
Agent score
90%

What's inside Lettuce

  1. Understand Lettuce API types

    main

    Lettuce is a scalable, thread-safe Redis client built on Netty and Project Reactor. It provides three distinct API styles for interacting with Redis:

    1. Synchronous API: For standard blocking operations.
    2. Asynchronous API: For non-blocking operations using CompletableFuture.
    3. Reactive API: For non-blocking, event-driven operations using Project Reactor (Flux and Mono).
  2. Understand Pipelining in Lettuce

    main

    Lettuce is a non-blocking, asynchronous client designed to operate using pipelining by default. This means multiple commands can be sent to the Redis server without waiting for the replies of previous commands.

    Key characteristics:

    • Thread Safety: Multiple threads can share a single connection. While one thread is waiting for a response, other threads can continue to send new commands.
    • Decoupled I/O: Built on Netty, Lettuce decouples reading from writing, allowing commands to be written and read independently but in sequence.
    • Async API Behavior: Every invocation on the async API returns a RedisFuture immediately after the command is written to the Netty pipeline, even before it reaches the underlying transport.

    Important Constraints for Shared Connections:

    • Long-running commands: If a command takes a long time to process, other threads using the same connection will wait longer for their results.
    • Transactional commands: Do not use MULTI on a shared connection.
    • Blocking commands: Avoid using Redis-blocking commands (e.g., BLPOP) on a shared connection, as they will block all other invocations on that connection. Use dedicated connections for blocking operations.
  3. Understand Lettuce command execution consistency levels

    main

    Lettuce provides two levels of consistency for Redis command execution. Choosing between them affects performance and reliability:

    • at-most-once execution: Commands may be lost. This is the highest performance option with the least overhead because it does not track command state or require retries.
    • at-least-once execution: Commands are guaranteed to be executed (with some exceptions). This uses a retry mechanism to counter transport losses, meaning commands may be duplicated but not lost. This requires more resources as commands are buffered during failures.

    Note on Message Ordering: Regardless of the consistency level, Lettuce guarantees that commands sent by a specific thread are not executed out-of-order (e.g., if C1 is executed, it must be executed before C2 from the same thread).

  4. Add Netty Native Transport dependencies

    main

    To use native transports, add the following Maven dependencies based on your platform and requirements:

    Linux epoll (x86_64)

    Requires Netty version 4.0.26.Final or higher.

    Linux io_uring (x86_64)

    Requires Netty version 4.1.54.Final or higher. This transport is experimental.

    macOS kqueue (x86_64)

    Requires Netty version 4.1.11.Final or higher.

    <!-- Linux epoll -->
    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-transport-native-epoll</artifactId>
        <version>${netty-version}</version>
        <classifier>linux-x86_64</classifier>
    </dependency>
    
    <!-- Linux io_uring -->
    <dependency>
        <groupId>io.netty.incubator</groupId>
        <artifactId>netty-incubator-transport-native-io_uring</artifactId>
        <version>0.0.1.Final</version>
        <classifier>linux-x86_64</classifier>
    </dependency>
    
    <!-- macOS kqueue -->
    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-transport-native-kqueue</artifactId>
        <version>${netty-version}</version>
        <classifier>osx-x86_64</classifier>
    </dependency>
  5. Manage TLS and mTLS certificates in tests

    main

    When TLS_ENABLED=yes is set in the Redis container, it generates certificates in the /redis/work/tls/ directory. These are surfaced on the host via the work/ volume mount. Tests read these files from <TEST_WORK_FOLDER>/work/tls/.

    Generated Files

    • ca.crt, ca.key: Self-signed test CA.
    • redis.crt, redis.key: Server certificate (CN localhost).
    • <cn>.crt, <cn>.key, <cn>.p12: Client certificates for each CN specified in TLS_CLIENT_CNS.

    mTLS Usage

    For mTLS scenarios, the PKCS#12 keystores (*.p12) use the password changeit. The TlsSettings class builds a truststore from these files. The ClientCertificate enum maps specific .p12 files (e.g., Client-test-cert.p12, client.p12) to various ACL/no-ACL test scenarios.

  6. Use RedisJSON in Power-user mode

    main

    Power-user mode is designed for applications that perform little to no processing on the Java layer. In this mode, you work with unprocessed JSON documents that have not undergone deserialization or serialization.

    This mode is achieved by using the JsonValue type directly, which can be used in commands like jsonGet and jsonSet without manual parsing.

    JsonPath myPath = JsonPath.of("$..mountain_bikes");
    RedisURI redisURI = RedisURI.Builder.redis("127.0.0.1").withPort(16379).build();
    try (RedisClient client = RedisClient.create(redisURI)) {
        RedisAsyncCommands<String, String> redis = client.connect().async();
        RedisFuture<List<JsonValue>> bikes = redis.jsonGet("bikes:inventory", myPath);
    
        CompletionStage<RedisFuture<String>> stage = bikes.thenApply(
                fetchedBikes -> redis.jsonSet("service_bikes", JsonPath.ROOT_PATH, fetchedBikes.get(0)));
    
        String result = stage.toCompletableFuture().get().get();
    }
  7. Refresh Redis Cluster topology view

    main

    If the cluster configuration changes (e.g., new nodes added, master migration), you may encounter frequent MOVED or ASK redirects. While Lettuce handles these transparently up to 5 times, you can manually or automatically refresh the topology view. The topology is bound to the RedisClusterClient instance.

    Ways to refresh:

    1. Manual: Call RedisClusterClient.reloadPartitions().
    2. Periodic Updates: Configure a background interval via ClusterTopologyRefreshOptions.
    3. Adaptive Updates: Configure background updates triggered by persistent disconnects or MOVED/ASK redirections via ClusterTopologyRefreshOptions.

    Background topology updating starts with the first connection obtained through the RedisClusterClient.

  8. Enable Brave Tracing for Redis commands

    main

    Lettuce supports Brave tracing, creating a span for every Redis command.

    Configuration Options

    • serviceName: The name of the service (defaults to redis).
    • Endpoint customizer: Used with a custom SocketAddressResolver to attach custom endpoint details.
    • Span customizer: Allows customization of spans based on the Command object.
    • Argument inclusion: You can choose to exclude command arguments from span tags (default is to include all).

    Prerequisites

    Add brave (at least 5.1) to your Maven pom.xml:

    <dependency>
        <groupId>io.zipkin.brave</groupId>
        <artifactId>brave</artifactId>
    </dependency>
    brave.Tracing clientTracing = …;
    
    BraveTracing tracing = BraveTracing.builder().tracing(clientTracing)
        .excludeCommandArgsFromSpanTags()
        .serviceName("custom-service-name-goes-here")
        .spanCustomizer((command, span) -> span.tag("cmd", command.getType().name()))
        .build();
    
    ClientResources resources = ClientResources.builder().tracing(tracing).build();
  9. Track command latency metrics using built-in latency tracking

    main

    Lettuce can track command execution metrics including execution count, latency to first response (min, max, percentiles), and latency to complete (min, max, percentiles). These metrics are published as CommandLatencyEvent objects on the client's EventBus.

    To consume these metrics, obtain the EventBus from the client's ClientResources and subscribe to the event stream, filtering for CommandLatencyEvent instances.

    RedisClient client = RedisClient.create();
    EventBus eventBus = client.getResources().eventBus();
    
    Subscription subscription = eventBus.get()
                    .filter(redisEvent -> redisEvent instanceof CommandLatencyEvent)
                    .cast(CommandLatencyEvent.class)
                    .subscribe(e -> System.out.println(e.getLatencies()));
  10. Use Spell Checking and Dictionaries

    main

    Provide query corrections for misspelled terms.

    • Spell Check: Use ftSpellcheck(index, term) to get corrections. Use SpellCheckArgs to configure distance (Levenshtein distance), terms("include", "dictionary"), and dialect(QueryDialects.DIALECT2).
    • Dictionary Management: Manage custom dictionaries for spell checking using ftDictadd(dict, terms...), ftDictdel(dict, term), and ftDictdump(dict) to retrieve all terms.
    • Synonym Management: Create synonym groups with ftSynupdate(index, group, terms...). Use SynUpdateArgs.skipInitialScan() to avoid reindexing existing documents.
    // Spell check
    SpellCheckArgs<String, String> spellArgs = SpellCheckArgs.<String, String>builder()
        .distance(2)
        .terms("include", "dictionary")
        .dialect(QueryDialects.DIALECT2)
        .build();
    List<SpellCheckResult<String>> results = search.ftSpellcheck("products-idx", "wireles hedphones", spellArgs);
    
    // Dictionary management
    search.ftDictadd("custom_dict", "smartphone", "tablet", "laptop");
    List<String> terms = search.ftDictdump("custom_dict");
    
    // Synonym management
    search.ftSynupdate("products-idx", "group1", "phone", "smartphone", "mobile");
    
    SynUpdateArgs synArgs = SynUpdateArgs.builder().skipInitialScan().build();
    search.ftSynupdate("products-idx", "group1", synArgs, "phone", "smartphone", "mobile", "cellphone");
  11. Use the Streaming API to process large Redis collections

    main

    When dealing with massive Redis collections that might exceed your application's heap memory, use Lettuce's StreamingChannel interfaces instead of returning full List, Set, or Map objects.

    Streaming methods allow Lettuce to push data to you as it arrives from Redis, enabling real-time processing while the command is still executing. The method call returns a long representing the total count of items processed.

    Available StreamingChannel types:

    • KeyStreamingChannel
    • ValueStreamingChannel
    • KeyValueStreamingChannel
    • ScoredValueStreamingChannel

    CRITICAL: Do not issue blocking calls (including synchronous Lettuce API calls) from inside a StreamingChannel callback. Doing so will block the EventLoop. If you need to perform further Redis operations within a callback, use the asynchronous or reactive APIs.

  12. Use Java Flight Recorder (JFR) for Lettuce events

    main

    Since version 6.1, Lettuce emits Connection and Cluster events as JFR events. The EventRecorder automatically handles this if the runtime provides the required JFR classes (JDK 8 update 262+).

    Supported JFR Events:

    • Connection: Attempt, Connect, Disconnect, Activated, Deactivated, Reconnect Attempt, Reconnect Failed.
    • Cluster: Topology Refresh initiated, Topology Changed, ASK and MOVED redirects.
    • Master/Replica: Sentinel Topology Refresh initiated, Master/Replica Topology Changed.

    To record these events, start your application with the JFR flag. To disable them, set the system property io.lettuce.core.jfr to false.