redis-rdb-cli

The tool provides two primary ways to move data between Redis instances.

rst (Continuous Sync)

rst migrates an initial RDB snapshot and then continuously syncs incremental data changes. It runs until manually stopped.

• Use case: Real-time synchronization or keeping a replica updated.
• Limitation: Only supports filtering by db index.

# Sync two instances
$ rst -s redis://127.0.0.1:6379 -m redis://127.0.0.1:6380 -r

# Sync a single instance to a Redis Cluster
$ rst -s redis://127.0.0.1:6379 -m redis://127.0.0.1:30001 -r -d 0

rmt (One-time Migration)

rmt performs a BGSAVE on the source to generate a snapshot, migrates that snapshot to the target, and then terminates.

• Use case: One-time data migration.

# Migrate RDB file to a remote Redis instance
$ rmt -s /path/to/dump.rdb -m redis://192.168.1.105:6379 -r

# Migrate RDB to a remote Redis Cluster using nodes.conf
$ rmt -s /path/to/dump.rdb -c ./nodes-30001.conf -r

There are two primary commands for data synchronization:

rmt (Migration)

Used for one-time migrations. It triggers a BGSAVE on the source Redis, migrates the resulting RDB snapshot to the target, and then terminates.

• Sync RDB to remote Redis: $ rmt -s <rdb_path> -m <redis_uri> -r
• Sync RDB to a Redis Cluster: $ rmt -s <rdb_path> -c <nodes_conf_path> -r

rst (Continuous Sync)

Used for continuous synchronization. It migrates the RDB snapshot and then continues to migrate incremental data. It does not terminate automatically; use CTRL+C to stop.

• Sync between two Redis instances: $ rst -s <source_uri> -m <target_uri> -r
• Sync single Redis to a Cluster: $ rst -s <source_uri> -m <target_uri> -r -d <db_index>

Note: rst only supports filtering by db (-d).

# Sync data between two Redis instances
$ rst -s redis://127.0.0.1:6379 -m redis://127.0.0.1:6380 -r

# Migrate an RDB file to a remote Redis instance
$ rmt -s /path/to/dump.rdb -m redis://192.168.1.105:6379 -r

When migrating data to a Redis Cluster using rst, be aware of the following constraints:

Cluster Stability Requirement

The tool uses the cluster's nodes.conf file and does not handle MOVED or ASK redirections. The cluster MUST be in a stable state:

• No slots should be in a migrating or importing state.
• No failovers (replica promotion) should occur during the process.

Unsupported Commands

The following commands are not supported during cluster migration:

• PUBLISH, SWAPDB, MOVE, FLUSHALL, FLUSHDB, MULTI, EXEC, SCRIPT FLUSH, SCRIPT LOAD, EVAL, EVALSHA.

Slot-Specific Command Constraints

The following commands are ONLY supported if all keys in the command belong to the same hash slot (e.g., using hash tags like del {user}:1 {user}:2):

• RPOPLPUSH, SDIFFSTORE, SINTERSTORE, SMOVE, ZINTERSTORE, ZUNIONSTORE, DEL, UNLINK, RENAME, RENAMENX, PFMERGE, PFCOUNT, MSETNX, BRPOPLPUSH, BITOP, MSET, COPY, BLMOVE, LMOVE, ZDIFFSTORE, GEOSEARCHSTORE, MSETEX.

You can monitor Redis servers (Standalone, Cluster, or Sentinel) using rmonitor and a Grafana dashboard.

Setup Steps:

1. Configure Metric Gateway: In redis-rdb-cli.conf, change metric_gateway from none to influxdb.
2. Start Dashboard: Navigate to the dashboard directory and run docker-compose up -d.
3. Run Monitor:
   • Standalone: rmonitor -s <uri> -n standalone
   • Cluster: rmonitor -s <uri> -n cluster
   • Sentinel: rmonitor -s <sentinel_uri> -n sentinel
4. View Results: Open http://localhost:3000/d/monitor/monitor in your browser and log in with admin/admin.

Note: Since version v0.1.9, rct -f mem results can also be displayed in Grafana.

# Start the monitoring dashboard
$ cd /path/to/redis-rdb-cli/dashboard
$ docker-compose up -d

# Monitor a standalone Redis instance
$ rmonitor -s redis://127.0.0.1:6379 -n standalone

You can create custom formatters to change how Redis data is outputted using the rct command.

Implementation Steps

1. Extend AbstractFormatterService: Implement the format() method and override specific event methods like applyString to define how different data types are processed.
2. Register via SPI: Create a file named com.moilioncircle.redis.rdb.cli.api.format.FormatterService in src/main/resources/META-INF/services/ containing your class name.
3. Deploy: Build the JAR with dependencies and copy it to the redis-rdb-cli/lib directory.
4. Run: Use the rct command.

Example Usage

$ rct -f test -s redis://127.0.0.1:6379 -o ./out.csv -t string -d 0 -e json

public class YourFormatterService extends AbstractFormatterService {
    @Override
    public String format() {
        return "test";
    }

    @Override
    public Event applyString(Replicator replicator, RedisInputStream in, int version, byte[] key, int type, ContextKeyValuePair context) throws IOException {
        byte[] val = new DefaultRdbValueVisitor(replicator).applyString(in, version);
        getEscaper().encode(key, getOutputStream());
        getEscaper().encode(val, getOutputStream());
        getOutputStream().write('\n');
        return context;
    }
}