The Raft Consensus Algorithm: How Distributed Databases Agree on Truth During Network Partitions

In a single-server architecture, consistency is trivial: the CPU executes transactions sequentially, and changes commit to a single disk. But in modern distributed systems—where Kubernetes clusters, CockroachDB, TiDB, and etcd span multiple physical availability zones—hardware fails constantly. Cables get severed, switches reboot, and network partitions isolate nodes. How do five independent servers agree on the exact order of financial transactions when they cannot reliably talk to one another? The answer is the Raft Consensus Algorithm.

1. The Problem with Paxos: The Need for Understandability

Before 2014, the distributed systems world relied almost exclusively on Leslie Lamport's Paxos algorithm. While mathematically brilliant, Paxos was notoriously opaque and difficult to implement in production without introducing subtle edge-case data corruption bugs. Stanford researchers Diego Ongaro and John Ousterhout designed Raft with a primary architectural goal: Understandability without sacrificing formal correctness.

2. The Three States of a Raft Node

At any given millisecond, every node in a Raft cluster exists in one of three states:

• Leader: The single authoritative node that receives all client writes, assigns sequential log indices, and replicates them to other nodes.
• Follower: Passive nodes that respond to incoming Remote Procedure Calls (RPCs) from the Leader.
• Candidate: A node that transitions to run for election if it stops hearing heartbeats from the Leader.

3. Leader Election and Randomized Heartbeat Timers

Raft divides time into arbitrary sequential Terms (Term 1, Term 2, etc.), acting as a logical clock. The cluster maintains consistency through periodic heartbeats (AppendEntries RPCs) sent by the Leader.

[Raft Leader Election Flow]
1. Leader crashes or network partition isolates it.
2. Follower's randomized election timer (150ms - 300ms) expires without a heartbeat.
3. Follower increments Term, votes for itself, and sends RequestVote RPC to all nodes.
4. If it receives votes from a strict majority (Quorum = N/2 + 1), it becomes the new Leader!

The stroke of genius in Raft is Randomized Election Timeouts. By assigning each follower a random countdown timer between 150ms and 300ms, nodes almost never start elections simultaneously, preventing 'Split-Vote' deadlocks where nobody wins a majority.

4. Log Replication and the Quorum Rule

When a client issues a write command (e.g. SET account_balance = 500):

1. The Leader appends the entry to its local log.
2. The Leader sends AppendEntries RPCs to all Followers.
3. Once a strict majority (Quorum: at least 3 out of 5 nodes) has written the entry to non-volatile disk, the Leader commits the transaction and responds to the client.
4. The Leader notifies followers in the next heartbeat to commit the entry to their local state machines.

5. Surviving Network Partitions: Preventing Split-Brain Disasters

Consider a 5-node cluster split across two data centers by a fiber optic line break: Nodes {A, B} on the East Coast and Nodes {C, D, E} on the West Coast.

If Node A was the original Leader in the minority partition {A, B}, it can receive client writes, but it can only replicate them to Node B (total 2 votes). Because 2 votes is less than the required majority ($5/2 + 1 = 3$), the entry is never committed, and the client receives a timeout.

Meanwhile, in the majority partition {C, D, E}, Node C times out, holds an election, secures 3 votes, and becomes the new legitimate Leader for Term 2. When the fiber line is repaired and the cluster reunites, Node A sees that Node C has a higher Term number. Node A immediately steps down to a Follower, uncommitted writes are rolled back, and the cluster resumes unified consensus without a single byte of corrupted state.

Frequently Asked Questions

Why do Raft clusters always have an odd number of nodes (3, 5, or 7)?

Consensus requires a strict majority quorum ($N/2 + 1$). A 3-node cluster can tolerate 1 failure. A 4-node cluster also requires 3 nodes for a majority, so it still only tolerates 1 failure while consuming more hardware and network traffic. Therefore, odd numbers give the optimal balance of fault tolerance and efficiency.

What is the difference between Raft and Byzantine Fault Tolerance (BFT)?

Raft assumes a 'Crash-Fault-Tolerant' (CFT) model where nodes may crash or delay messages, but they do not lie or send intentionally malicious data. Blockchain networks use Byzantine Fault Tolerance (BFT) to defend against adversarial, dishonest participants.