Redix

repository·main·Indexed 22 days ago

https://github.com/whatyouhide/redix

A high-performance, resilient Redis and Valkey client for Elixir. Redix provides an idiomatic interface for sending commands, pipelining, and handling Pub/Sub. It includes support for Redis Sentinel and Redis Cluster, featuring a Cluster Manager for topology discovery, hash slot connection routing, and replica read configuration.

Tokens
10.1K
Snippets
35
Records
47
Agent score
73%

What's inside Redix

  1. How Redix telemetry and instrumentation work

    main
    Redix uses the Telemetry library for instrumentation and extensible logging. It works by publishing events through a common interface, which can then be intercepted by attached handlers. To see the specific list of events emitted by Redix, refer to the Redix.Telemetry module.
  2. How Redix handles reconnections

    main

    Redix automatically attempts to reconnect to the Redis server if the connection drops.

    Error Handling

    If a disconnection occurs while requests are pending, Redix functions return {:error, %Redix.ConnectionError{reason: :disconnected}}. The caller is responsible for retrying these requests.

    Backoff Strategy

    Redix uses an exponential backoff strategy for reconnection attempts:

    1. The first attempt occurs after an interval defined by :backoff_initial.
    2. Subsequent attempts increase the interval by a fixed factor of 1.5 (e.g., n * 1.5, n * 1.5 * 1.5, etc.).
    3. To prevent the interval from growing indefinitely, use the :backoff_max option to cap the delay.

    Tip: To simulate a constant reconnection interval (e.g., every 5 seconds), set both :backoff_initial and :backoff_max to the same value (e.g., 5_000).

    # Example: Constant reconnection interval of 5 seconds
    Redix.start_link(..., backoff_initial: 5_000, backoff_max: 5_000)
  3. Caveats of name-based connection pools

    main

    When using a name-based pool for Redix, be aware of the following limitations:

    1. Unintelligent Load Distribution: The load is distributed among connections (e.g., via random selection), but it is not "smart." It does not account for connection health or latency, meaning it won't automatically avoid sending requests to slower connections.
    2. Ordering and Race Conditions: Because commands are sent over different connections, you cannot guarantee the order of execution if commands are issued from different processes. While a single process will block until it receives a reply (ensuring sequential execution for that process), concurrent processes using the pool may have their commands processed by Redis in a different order than they were issued.
  4. Write a custom telemetry handler for Redix

    main

    You can create a custom module to control how Redix events are logged or processed. A handler must implement a function (typically handle_event/4) that matches the Telemetry signature.

    When handling Redix events, the event parameter will be one of the following:

    • :disconnection: Triggered when a connection is lost. Metadata includes address and reason (an exception).
    • :failed_connection: Triggered when a connection attempt fails. Metadata includes address and reason (an exception).
    • :connection: Triggered on successful connection or reconnection. Metadata includes address.
    defmodule MyApp.RedixTelemetryHandler do
      require Logger
    
      def handle_event([:redix, event], _measurements, metadata, _config) do
        case event do
          :disconnection ->
            human_reason = Exception.message(metadata.reason)
            Logger.warn("Disconnected from #{metadata.address}: #{human_reason}")
    
          :failed_connection ->
            human_reason = Exception.message(metadata.reason)
            Logger.warn("Failed to connect to #{metadata.address}: #{human_reason}")
    
          :connection ->
            Logger.debug("Connected/reconnected to #{metadata.address}")
        end
      end
    end
  5. Install Redix

    main

    To use Redix in your Elixir project, add :redix to your mix.exs dependencies. If you require SSL connections, it is recommended to also include the :castore dependency.

    After updating mix.exs, run mix deps.get to fetch the dependencies.

    defp deps do
      [
        {:redix, "~> 1.1"},
        {:castore, ">= 0.0.0"}
      ]
    end
  6. Use a single named Redix instance

    main

    For many applications, a single global Redix instance is sufficient, especially when Redis requests do not map one-to-one to user requests. You can start a named Redix process under your supervision tree and then access it from anywhere in your application using its registered name.

    You can also use multiple named instances to separate different types of traffic (e.g., one instance for large, infrequent requests and another for short, frequent requests).

    # In your supervision tree
    children = [
      {Redix, name: :redix}
    ]
    
    # Using the named instance anywhere in your app
    Redix.command(:redix, ["PING"])
    #=> {:ok, "PONG"}
  7. Read from replicas in Redix.Cluster

    main

    By default, all commands are routed to primary nodes. To enable reading from replicas, you must:

    1. Start the cluster with read_from_replicas: true.
    2. Use the :route option in Redix.Cluster.command/3 or Redix.Cluster.pipeline/3.

    Available routing options:

    • :replica: Read from a replica for the key's slot, failing if none are reachable.
    • :prefer_replica: Prefer a replica but fall back to the primary if no replica is reachable.
    Redix.Cluster.start_link(
      name: :my_cluster,
      nodes: ["redis://localhost:7000"],
      read_from_replicas: true
    )
    
    # Read from a replica, failing if none are reachable
    Redix.Cluster.command(:my_cluster, ["GET", "mykey"], route: :replica)
    
    # Prefer a replica, fallback to primary
    Redix.Cluster.command(:my_cluster, ["GET", "mykey"], route: :prefer_replica)
  8. Implement a name-based connection pool

    main

    When high-load support is required, you can implement a connection pool by starting multiple Redix connections registered with unique names (e.g., :redix_0, :redix_1) under a supervisor.

    To use this pattern, create a wrapper module that selects a connection from the pool (for example, using a random strategy) and forwards commands to it. This allows you to distribute the load across multiple TCP streams.

    ```elixir
    defmodule MyApp.Redix do
      @pool_size 5
    
      def child_spec(_args) do
        # Specs for the Redix connections.
        children =
          for index <- 0..(@pool_size - 1) do
            Supervisor.child_spec({Redix, name: :
  9. Use Redix.Cluster for Redis Cluster support

    main

    Redix supports Redis Cluster via the Redix.Cluster module. It transparently handles routing commands to the correct nodes based on hash slots, manages MOVED and ASK redirections during resharding, maintains the cluster topology map, and splits pipelines across nodes.

    To start a cluster connection, use Redix.Cluster.start_link/1 with a unique name and a list of seed nodes.

    # Start a cluster connection with a name and one or more seed nodes.
    {:ok, _pid} = Redix.Cluster.start_link(
      name: :my_cluster,
      nodes: ["redis://localhost:7000", "redis://localhost:7001"]
    )
    
    # Issue commands across the cluster.
    Redix.Cluster.command(:my_cluster, ["SET", "mykey", "myvalue"])
    #=> {:ok, "OK"}
    
    Redix.Cluster.command(:my_cluster, ["GET", "mykey"])
    #=> {:ok, "myvalue"}
  10. Attach a custom Redix telemetry handler

    main

    To activate your custom handler, use :telemetry.attach_many/4. You must provide a unique name for the handler, a list of the specific Redix events you want to listen to, and the function to call when those events occur.

    events = [
      [:redix, :disconnection],
      [:redix, :failed_connection],
      [:redix, :connection]
    ]
    
    :telemetry.attach_many(
      "my-redix-log-handler",
      events,
      &MyApp.RedixTelemetryHandler.handle_event/4,
      :config_not_needed_here
    )
  11. Handle vital Redis dependencies in supervision trees

    main

    If your application cannot function without Redis, you should use a combination of :sync_connect and :exit_on_disconnection to ensure your application's state remains consistent with the availability of Redis.

    1. Set sync_connect: true to ensure the application doesn't start without an initial connection.
    2. Set exit_on_disconnection: true so the Redix process crashes if the connection is lost later.
    3. Place the Redix process and its dependent services under a supervisor using a strategy like :rest_for_one. This ensures that if Redix crashes, the dependent services are also brought down, preventing the app from running in a broken state.

    Example Supervision Tree:

    # Define children where Redix and its dependents are isolated
    isolated_children = [
      {Redix, sync_connect: true, exit_on_disconnection: true},
      MyApp.MyGenServer
    ]
    
    # Use :rest_for_one so if Redix fails, MyGenServer is also restarted/stopped
    isolated_supervisor = %{
      id: MyChildSupervisor,
      type: :supervisor,
      start: {Supervisor, :start_link, [isolated_children, [strategy: :rest_for_one]]},
    }
    
    # Main application tree
    children = [
      MyApp.Child1,
      isolated_supervisor,
      MyApp.Child2
    ]
    
    Supervisor.start_link(children, strategy: :one_for_one)
  12. How Redix Pub/Sub handles reconnections and subscriptions

    main

    The Redix.PubSub.Connection manages Redis Pub/Sub subscriptions using a state machine. When a connection is lost, the handler enters a :disconnected state.

    Key behaviors include:

    • Automatic Resubscription: Upon successful reconnection, the connection automatically resubscribes to all previously active channels and patterns that still have active subscribers in memory.
    • Subscriber Monitoring: The connection monitors the PIDs of subscribers. If a subscriber process goes down while the connection is disconnected, it is removed from the internal subscription list to prevent unnecessary resubscriptions upon reconnection.
    • State Transitions: The connection transitions between :connected and :disconnected states. During disconnection, it uses an exponential backoff strategy (controlled by :backoff_initial, :backoff_max, and an internal exponent of 1.5) to attempt reconnections.
    • Message Delivery: When a message is received from Redis, it is dispatched to the monitored subscriber PIDs via a message format: {:redix_pubsub, sender_pid, monitor_ref, kind, properties}.