How Redix telemetry and instrumentation work
mainRedix.Telemetry module.repository·main·Indexed 22 days ago
https://github.com/whatyouhide/redixA high-performance, resilient Redis and Valkey client for Elixir. Redix provides an idiomatic interface for sending commands, pipelining, and handling Pub/Sub. It includes support for Redis Sentinel and Redis Cluster, featuring a Cluster Manager for topology discovery, hash slot connection routing, and replica read configuration.
Redix.Telemetry module.Redix automatically attempts to reconnect to the Redis server if the connection drops.
If a disconnection occurs while requests are pending, Redix functions return {:error, %Redix.ConnectionError{reason: :disconnected}}. The caller is responsible for retrying these requests.
Redix uses an exponential backoff strategy for reconnection attempts:
:backoff_initial.1.5 (e.g., n * 1.5, n * 1.5 * 1.5, etc.).:backoff_max option to cap the delay.Tip: To simulate a constant reconnection interval (e.g., every 5 seconds), set both :backoff_initial and :backoff_max to the same value (e.g., 5_000).
# Example: Constant reconnection interval of 5 seconds
Redix.start_link(..., backoff_initial: 5_000, backoff_max: 5_000)When using a name-based pool for Redix, be aware of the following limitations:
You can create a custom module to control how Redix events are logged or processed. A handler must implement a function (typically handle_event/4) that matches the Telemetry signature.
When handling Redix events, the event parameter will be one of the following:
:disconnection: Triggered when a connection is lost. Metadata includes address and reason (an exception).:failed_connection: Triggered when a connection attempt fails. Metadata includes address and reason (an exception).:connection: Triggered on successful connection or reconnection. Metadata includes address.defmodule MyApp.RedixTelemetryHandler do
require Logger
def handle_event([:redix, event], _measurements, metadata, _config) do
case event do
:disconnection ->
human_reason = Exception.message(metadata.reason)
Logger.warn("Disconnected from #{metadata.address}: #{human_reason}")
:failed_connection ->
human_reason = Exception.message(metadata.reason)
Logger.warn("Failed to connect to #{metadata.address}: #{human_reason}")
:connection ->
Logger.debug("Connected/reconnected to #{metadata.address}")
end
end
endTo use Redix in your Elixir project, add :redix to your mix.exs dependencies. If you require SSL connections, it is recommended to also include the :castore dependency.
After updating mix.exs, run mix deps.get to fetch the dependencies.
defp deps do
[
{:redix, "~> 1.1"},
{:castore, ">= 0.0.0"}
]
endFor many applications, a single global Redix instance is sufficient, especially when Redis requests do not map one-to-one to user requests. You can start a named Redix process under your supervision tree and then access it from anywhere in your application using its registered name.
You can also use multiple named instances to separate different types of traffic (e.g., one instance for large, infrequent requests and another for short, frequent requests).
# In your supervision tree
children = [
{Redix, name: :redix}
]
# Using the named instance anywhere in your app
Redix.command(:redix, ["PING"])
#=> {:ok, "PONG"}By default, all commands are routed to primary nodes. To enable reading from replicas, you must:
read_from_replicas: true.:route option in Redix.Cluster.command/3 or Redix.Cluster.pipeline/3.Available routing options:
:replica: Read from a replica for the key's slot, failing if none are reachable.:prefer_replica: Prefer a replica but fall back to the primary if no replica is reachable.Redix.Cluster.start_link(
name: :my_cluster,
nodes: ["redis://localhost:7000"],
read_from_replicas: true
)
# Read from a replica, failing if none are reachable
Redix.Cluster.command(:my_cluster, ["GET", "mykey"], route: :replica)
# Prefer a replica, fallback to primary
Redix.Cluster.command(:my_cluster, ["GET", "mykey"], route: :prefer_replica)When high-load support is required, you can implement a connection pool by starting multiple Redix connections registered with unique names (e.g., :redix_0, :redix_1) under a supervisor.
To use this pattern, create a wrapper module that selects a connection from the pool (for example, using a random strategy) and forwards commands to it. This allows you to distribute the load across multiple TCP streams.
```elixir
defmodule MyApp.Redix do
@pool_size 5
def child_spec(_args) do
# Specs for the Redix connections.
children =
for index <- 0..(@pool_size - 1) do
Supervisor.child_spec({Redix, name: :Redix supports Redis Cluster via the Redix.Cluster module. It transparently handles routing commands to the correct nodes based on hash slots, manages MOVED and ASK redirections during resharding, maintains the cluster topology map, and splits pipelines across nodes.
To start a cluster connection, use Redix.Cluster.start_link/1 with a unique name and a list of seed nodes.
# Start a cluster connection with a name and one or more seed nodes.
{:ok, _pid} = Redix.Cluster.start_link(
name: :my_cluster,
nodes: ["redis://localhost:7000", "redis://localhost:7001"]
)
# Issue commands across the cluster.
Redix.Cluster.command(:my_cluster, ["SET", "mykey", "myvalue"])
#=> {:ok, "OK"}
Redix.Cluster.command(:my_cluster, ["GET", "mykey"])
#=> {:ok, "myvalue"}To activate your custom handler, use :telemetry.attach_many/4. You must provide a unique name for the handler, a list of the specific Redix events you want to listen to, and the function to call when those events occur.
events = [
[:redix, :disconnection],
[:redix, :failed_connection],
[:redix, :connection]
]
:telemetry.attach_many(
"my-redix-log-handler",
events,
&MyApp.RedixTelemetryHandler.handle_event/4,
:config_not_needed_here
)If your application cannot function without Redis, you should use a combination of :sync_connect and :exit_on_disconnection to ensure your application's state remains consistent with the availability of Redis.
sync_connect: true to ensure the application doesn't start without an initial connection.exit_on_disconnection: true so the Redix process crashes if the connection is lost later.:rest_for_one. This ensures that if Redix crashes, the dependent services are also brought down, preventing the app from running in a broken state.Example Supervision Tree:
# Define children where Redix and its dependents are isolated
isolated_children = [
{Redix, sync_connect: true, exit_on_disconnection: true},
MyApp.MyGenServer
]
# Use :rest_for_one so if Redix fails, MyGenServer is also restarted/stopped
isolated_supervisor = %{
id: MyChildSupervisor,
type: :supervisor,
start: {Supervisor, :start_link, [isolated_children, [strategy: :rest_for_one]]},
}
# Main application tree
children = [
MyApp.Child1,
isolated_supervisor,
MyApp.Child2
]
Supervisor.start_link(children, strategy: :one_for_one)The Redix.PubSub.Connection manages Redis Pub/Sub subscriptions using a state machine. When a connection is lost, the handler enters a :disconnected state.
Key behaviors include:
:connected and :disconnected states. During disconnection, it uses an exponential backoff strategy (controlled by :backoff_initial, :backoff_max, and an internal exponent of 1.5) to attempt reconnections.{:redix_pubsub, sender_pid, monitor_ref, kind, properties}.