redis4cats

repository·series/2.x·Indexed 18 days ago

https://github.com/profunktor/redis4cats

A Scala Redis client built on top of Cats Effect, FS2, and the Lettuce async Java client. It provides a functional interface for interacting with Redis, supporting standard effectful commands and streaming operations. The library includes support for Redis Master-Replica setups, Sentinel clusters, ACL management via AclCommands, and TLS configuration.

Tokens
32.2K
Snippets
85
Records
116
Agent score
63%

What's inside redis4cats

  1. Overview of the experimental Streams API

    series/2.x

    The redis4cats library provides an experimental API for interacting with Redis at the stream level using Stream[F[_], A] built on top of fs2. This API is divided into two main functional areas:

    1. PubSub: A simple, safe, and pure functional streaming client for interacting with Redis PubSub.
    2. Streams: A high-level, safe, and pure functional API for interacting with Redis Streams.
  2. Use the Effects API for Redis operations

    series/2.x

    The Effects API in redis4cats allows you to perform Redis operations within a functional effect type F[_]. This API is built on top of cats-effect, meaning all operations are asynchronous and non-blocking, returning values wrapped in your chosen effect type (typically IO).

    Instead of working with raw commands, you use specialized APIs organized by Redis data types and features. Common API groups include:

    • Data Types: Hashes, Lists, Sets, Sorted Sets, Strings, Bitmaps, and HyperLogLog.
    • Specialized Features: Geo (geospatial), JSON, and Scripting (Lua).
    • Management: Keys, Connection, ACL (Access Control Lists), and Server management.
  3. How pipelining works in redis4cats

    series/2.x

    Pipelining allows you to speed up queries by disabling Redis's default autoflush mode. Instead of waiting for a response for every command, you can send a batch of commands to the server and flush them all at once.

    In redis4cats, pipelining is modeled as a Resource with the following lifecycle:

    • Acquire: Disables autoflush and sends a List[F[Unit]] of commands.
    • Release: Flushes the commands on success, or logs an error if the operation fails or is cancelled.
    • Guarantee: Re-enables autoflush.

    ⚠️ Important Constraint: Pipelining shares the same asynchronous implementation as transactions. This means you can only run sequential pipelines from a single RedisCommands instance; you cannot run them concurrently on the same instance.

  4. Choose between JsonValue and String variants

    series/2.x

    The RedisJSON API provides two variants for most methods:

    1. JsonValue variants: These work with Lettuce's JsonValue type (e.g., jSet, jGet, arrAppend).
    2. String variants: These accept or return raw JSON strings and are identified by the Str or Raw suffix (e.g., jSetStr, jGetRaw, arrAppendStr).

    Recommendation: For most use cases, use the String variants. They allow you to work directly with JSON text without requiring the client to perform JSON parsing.

  5. How the ACL API works

    series/2.x

    The ACL API provides a purely functional interface for managing Redis Access Control Lists (ACLs), including users, passwords, and permissions for commands, keys, and channels.

    The API is represented by the AclCommands[F] algebra, which is a union of two sub-algebras:

    1. AclManagement[F]: Server-wide operations like WHOAMI, CAT, GENPASS, LIST, LOAD, SAVE, and LOG.
    2. AclUserManagement[F]: Per-user operations like USERS, GETUSER, SETUSER, and DELUSER.

    Note that AclCommands does not take type parameters for key/value codecs (like K/V) because ACL operations work on usernames and rule strings rather than user-defined data types.

    import cats.effect.{IO, Resource}
    import dev.profunktor.redis4cats.Redis
    import dev.profunktor.redis4cats.algebra.AclCommands
    import dev.profunktor.redis4cats.data._
    
    val commandsApi: Resource[IO, AclCommands[IO]] = {
      Redis[IO].fromClient[String, String](null, null.asInstanceOf[RedisCodec[String, String]]).widen[AclCommands[IO]]
    }
  6. Authenticate with Redis using RedisCredentials

    series/2.x

    Authentication can be handled via the URI string, by attaching credentials to a RedisURI using .withCredentials, or via RedisUriConfig.

    RedisCredentials supports two modes:

    • RedisCredentials.Password(token): For AUTH <token> (no username).
    • RedisCredentials.UsernameAndPassword(username, token): For Redis 6+ ACL-style AUTH <username> <token>.
    import dev.profunktor.redis4cats.connection._
    
    // Token without a username
    RedisURI.make[IO]("redis://localhost:6379").map(_.withCredentials(RedisCredentials.Password(token)))
    
    // Username + token (Redis 6 ACL style)
    RedisURI
      .make[IO]("redis://localhost:6379")
      .map(_.withCredentials(RedisCredentials.UsernameAndPassword(username, token)))
  7. Understand RedisCodec and basic types

    series/2.x

    Redis is a key-value store that typically stores values in a string-like format. Redis4Cats uses the RedisCodec[K, V] type to parameterize the types of keys (K) and values (V) used in commands.

    Commonly used built-in codecs include:

    • RedisCodec.Utf8 (most common)
    • RedisCodec.Ascii
    • RedisCodec.Bytes
  8. Use RedisPipe for pipelining commands

    series/2.x

    While you can manually manage autoflush and flush commands via the RedisCommands API, it is highly recommended to use the pipeline or pipeline_ methods. These methods handle the complex resource management and autoflush toggling for you.

    Key Requirements for Pipeline Commands:

    1. Asynchronous Execution: Every command must be forked using .start because commands are sent to the server asynchronously and no response is received until the batch is flushed.
    2. No Sequencing: You cannot use flatMap to sequence commands that are part of a pipeline. Every command must be atomic and independent of the results of previous commands in the same pipeline.

    Choosing between pipeline and pipeline_:

    • Use pipeline(ops) if you need a TxStore to store and retrieve values produced within the pipeline.
    • Use pipeline_(ops) if you do not need a store, as it is simpler.
    // Example using pipeline with a TxStore
    val ops = (store: TxStore[IO, String, Option[String]]) =>
      List(
        redis.set(key1, "osx"),
        redis.get(key3).flatMap(store.set(key3)),
        redis.set(key2, "linux")
      )
    
    val runPipeline = redis.pipeline(ops)
  9. Migrate redis4cats to sbt 2.0

    series/2.x

    To migrate the project from sbt 1.x to sbt 2.0, follow these high-level steps once sbt 2.0.0 final is released:

    1. Update sbt version: Change project/build.properties to use sbt.version=2.0.0.
    2. Update Environment: Ensure the CI and local environments use JDK 17+, as sbt 2.0 requires it for the metabuild.
    3. Update Plugins: Upgrade plugins to their sbt 2.x-compatible versions. Note that some plugins (like sbt-tpolecat, sbt-microsites, sbt-site, and sbt-prompt) may be blocked or require manual management/replacement.
    4. Simplify Settings: In sbt 2.0, bare settings apply to all subprojects. You can optionally remove the ThisBuild / prefix from settings like scalaVersion, crossScalaVersions, organization, etc.
    5. Clean up Meta-build Files: Delete redundant Metals files (project/metals.sbt, etc.) and review project/plugins.sbt for unnecessary resolvers or library dependency schemes.
    6. Verify: Run sbt compile, sbt +test, sbt mimaReportBinaryIssuesIfRelevant, and sbt doc to ensure the build is stable.
  10. Establish a Master/Replica connection

    series/2.x

    Master/Replica connections use RedisMasterReplica. This abstraction manages connections to both master and replica nodes. You can specify how to read from the cluster using ReadFrom (e.g., ReadFrom.UpstreamPreferred).

    import cats.effect.{IO, Resource}
    import cats.implicits._
    import dev.profunktor.redis4cats.Redis
    import dev.profunktor.redis4cats.algebra.StringCommands
    import dev.profunktor.redis4cats.connection.RedisMasterReplica
    import dev.profunktor.redis4cats.data.ReadFrom
    
    val commands: Resource[IO, StringCommands[IO, String, String]] =
      for {
        uri <- Resource.eval(RedisURI.make[IO]("redis://localhost"))
        conn <- RedisMasterReplica[IO].make(RedisCodec.Utf8, uri)(ReadFrom.UpstreamPreferred.some)
        redis <- Redis[IO].masterReplica(conn)
      } yield redis
    
    commands.use { redis =>
      redis.set("foo", "123") >> IO.unit
    }
  11. Use Bitmaps commands

    series/2.x

    Once you have a BitCommands[F, K, V] instance, you can perform various bitmap operations such as setting/getting bits, performing bitwise OR operations, and using complex bitfield operations.

    Common operations include:

    • setBit(key, offset, value): Sets the bit at the specified offset to the given value.
    • getBit(key, offset): Retrieves the bit value at the specified offset.
    • bitOpOr(dest, src1, src2): Performs a bitwise OR operation between two keys and stores the result in a destination key.
    • bitField(key, *operations): Executes multiple bitfield operations (like SetUnsigned or IncrUnsignedBy) on a single key.
    import cats.effect.IO
    
    val testKey  = "foo"
    val testKey2 = "bar"
    val testKey3 = "baz"
    
    commandsApi.use { cmd => // BitCommands[IO, String, String]
      for {
        a <- cmd.setBit(testKey, 7, 1)
        _ <- cmd.setBit(testKey2, 7, 0)
        b <- cmd.getBit(testKey, 6)
        _ <- cmd.bitOpOr(testKey3, testKey, testKey2)
        bf <- cmd.bitField(
          "inmap",
          SetUnsigned(2, 1),
          SetUnsigned(3, 1),
          SetUnsigned(5, 1),
          SetUnsigned(10, 1),
          SetUnsigned(11, 1),
          SetUnsigned(14, 1),
          IncrUnsignedBy(14, 1)
        )
      } yield ()
    }
  12. Initialize the RedisJSON API

    series/2.x

    To use RedisJSON commands, you need to acquire a connection that is widened to the JsonCommands trait. You can do this by using Redis[IO].fromClient and applying .widen[JsonCommands[IO, K, V]] to the resulting client.

    In the example below, K and V represent the key and value types (e.g., String).

    import cats.effect.{IO, Resource}
    import cats.implicits._
    import dev.profunktor.redis4cats.Redis
    import dev.profunktor.redis4cats.algebra.JsonCommands
    import dev.profunktor.redis4cats.data._
    
    // Example: Initializing a JsonCommands API for String keys and String values
    val commandsApi: Resource[IO, JsonCommands[IO, String, String]] = {
      Redis[IO].fromClient[String, String](null, null.asInstanceOf[RedisCodec[String, String]])
        .widen[JsonCommands[IO, String, String]]
    }