Leader Election
The process by which distributed nodes agree on a single coordinator to serialize decisions, using consensus (Raft, Paxos, ZooKeeper) and safeguards against split-brain and failover gaps.
By Ratish Jain · Updated
Definition
Leader election is the process by which a group of distributed nodes agrees on a single node — the leader or coordinator — responsible for making decisions the others follow, such as ordering writes or assigning work.
At any time there should be exactly one leader. If the leader fails, the remaining nodes must detect the failure and elect a new one, a process called failover.
Why It Exists
Many coordination tasks are far simpler with a single decision-maker: serializing writes, assigning partitions, running scheduled jobs, or maintaining a single source of truth. Letting every node decide independently invites conflicts and divergence.
But hard-coding one node as leader creates a single point of failure. Leader election gives the benefits of a coordinator while keeping the system fault-tolerant — the role can move to a healthy node automatically when the current leader dies.
How It Works
Election relies on consensus: nodes must agree on the leader even amid failures and message delays. Raft uses randomized election timeouts — a node that times out becomes a candidate, requests votes, and wins with a majority quorum, then sends heartbeats to stay leader. Paxos and ZAB (ZooKeeper) achieve the same with different mechanics. Systems often outsource this to a coordination service like ZooKeeper, etcd, or Consul via ephemeral nodes or leases.
The central danger is split-brain: a network partition making two nodes each believe they are leader, causing conflicting writes. Requiring a majority (so only one side can win), plus fencing tokens (monotonic epoch numbers that let storage reject a stale leader), and lease timeouts prevent it. On leader failure, heartbeats stop, followers time out, and a new election runs.
What It's Used For
- Choosing the primary in single-leader replication (databases, Kafka controller).
- Assigning partition ownership and rebalancing in distributed systems.
- Running a singleton job or scheduler exactly once across a cluster.
- Coordinating configuration and service discovery via ZooKeeper/etcd/Consul.
- Automatic failover to a standby when the active coordinator dies.
Tradeoffs
- A single coordinator makes ordering and decisions simple and consistent.
- Automatic failover removes the leader as a permanent single point of failure.
- Consensus with majority quorum prevents two valid leaders at once.
- Election and failover add a window of unavailability while re-electing.
- Split-brain risk requires fencing tokens and majority quorums to avoid.
- The leader can become a throughput bottleneck for writes.
- Consensus protocols are subtle and hard to implement correctly.
When to Use / Avoid
- Use when
- Exactly one node must coordinate — ordering writes, owning partitions, or running a singleton task — while surviving failures.
- Avoid when
- Work is embarrassingly parallel and leaderless (quorum/CRDT) designs suffice, avoiding the leader bottleneck and election complexity.
- Real-world
- Kafka elects a controller (KRaft/ZooKeeper); etcd and Consul use Raft; Kubernetes leader-elects controller-manager instances.
Interview Questions
Split-brain is when a network partition leaves two nodes each believing they are leader, so both accept writes and the data diverges. Prevent it by requiring a majority quorum to win an election (only one side of a partition can hold a majority) and by using fencing tokens — monotonically increasing epoch numbers that let storage reject writes from a stale, deposed leader.
Each follower runs a randomized election timeout. If it hears no heartbeat before the timeout, it becomes a candidate, increments the term, votes for itself, and requests votes from others. A node grants at most one vote per term; the candidate that collects a majority becomes leader and sends heartbeats to reset timeouts. Randomized timeouts make split votes rare, and a new term triggers re-election if one occurs.
- →Leader election picks one coordinator to serialize decisions; failover replaces it.
- →Built on consensus (Raft, Paxos, ZAB) needing a majority quorum to agree.
- →Often delegated to ZooKeeper, etcd, or Consul via leases/ephemeral nodes.
- →Split-brain (two leaders) is prevented by majority quorums and fencing tokens.
- →Trade-offs: brief unavailability during re-election and a possible leader bottleneck.
Related
Everything on Archprimer is connected — follow the graph.