raft-rs

The raft-rs crate provides only the core Consensus Module. It does not include implementations for the Log, State Machine, or Transport layers. To build a complete distributed system, you must implement and integrate your own components for:

1. Log: Storage for Raft logs.
2. State Machine: Storage for user data.
3. Transport: The network layer for node communication.

The Consensus Module is designed to be customizable, flexible, and resilient, allowing it to be embedded into your specific architecture.

The raft crate supports two different ways to encode/decode gRPC messages. By default, it uses rust-protobuf.

To use Prost instead of the default rust-protobuf:

1. Disable default features.
2. Enable the prost-codec feature.

Benchmarks are implemented using Criterion. To run them, you must first install gnuplot.

Running Benchmarks

Execute cargo bench to run the suite. Results, including plots and charts, can be found in target/criterion/report/index.html.

Comparing Performance Between Branches

You can compare the performance of your current branch against master using baselines:

1. Checkout master and save the baseline.
2. Checkout your target branch and run the benchmark using the saved baseline.

# Install gnuplot first, then:
cargo bench

# To compare performance between two branches:
git checkout master
cargo bench --bench benches -- --save-baseline master
git checkout other
cargo bench --bench benches -- --baseline master

The core loop of a Raft application involves checking for pending work using has_ready(), retrieving that work via ready(), and then processing it.

1. Check for work: Call has_ready() to see if there are messages to send, entries to persist, or snapshots to create.
2. Retrieve work: Call ready() to get a Ready object. This object contains entries to be saved, snapshot data, hard_state to be persisted, and messages to be sent to peers.
3. Handle messages: Use take_messages() to get outbound messages.
4. Persist and Advance: After persisting the data (entries, snapshot, hard state), you must call on_persist_ready(number) and then advance(ready) to move the state machine forward. advance returns a LightReady containing committed_entries that can now be applied to your state machine.

For convenience, you can import the most commonly used types from the prelude module. This includes Config, Raft, Storage, RawNode, Ready, and others.

use raft::prelude::*;

To create a Raft node, use RawNode::new. You must provide a Config object, a Storage implementation (such as MemStorage), and a slog::Logger. It is recommended to call config.validate() before initializing the node.

use raft::{
    Config,
    storage::MemStorage,
    raw_node::RawNode,
};
use slog::{Drain, o};

// Select some defaults, then change what we need.
let config = Config {
    id: 1,
    ..Default::default()
};
// Initialize logger.
let logger = slog::Logger::root(slog_stdlog::StdLog.fuse(), o!());
// ... Make any configuration changes.
// After, make sure it's valid!
config.validate().unwrap();
// We'll use the built-in `MemStorage`, but you will likely want your own.
// Finally, create our Raft node!
let storage = MemStorage::new_with_conf_state((vec![1], vec![]));
let mut node = RawNode::new(&config, storage, &logger).unwrap();

Raft nodes require periodic driving to handle timeouts and elections. Use the tick() method on a RawNode at regular intervals (e.g., every 100ms) to advance the internal state machine.

// Inside a loop, drive Raft every 100ms
loop {
    // ... handle incoming events ...
    
    let elapsed = now.elapsed();
    if elapsed >= remaining_timeout {
        remaining_timeout = timeout;
        // We drive Raft every 100ms.
        node.tick();
    } else {
        remaining_timeout -= elapsed;
    }
}

The max_apply_unpersisted_log_limit field on RaftLog controls how many unpersisted entries can be applied to the state machine. This helps prevent the application from getting too far ahead of the persisted log.

Set this value on your RaftLog instance to tune the balance between performance and durability.

For testing purposes, MemStorage provides a thread-safe, in-memory implementation of the Storage trait. Note that MemStorage only contains Raft logs and does not store applied data.

To use it:

1. Create a new instance with MemStorage::new().
2. Initialize it with a configuration using initialize_with_conf_state.
3. Access the underlying MemStorageCore using .rl() (read lock) or .wl() (write lock) to manipulate logs or state directly.