Archprimer
ArchitectureAdvanced 6 min read

WhatsApp / Chat

A real-time messaging system built on persistent WebSocket connections, with message fanout, delivery/read receipts, and online presence at billions-of-messages-per-day scale.

WebSocketsKafkaFanoutMessagingPresence

By Ratish Jain · Updated

Diagram with 6 components: Mobile App, Conn LB, Chat Server, Message Bus, Session Store, Message DB. Connections: Mobile App → Conn LB (1 · connect); Conn LB → Chat Server (2 · route); Chat Server → Message Bus (3 · publish); Message Bus → Chat Server (4 · deliver); Chat Server → Session Store (who/where); Chat Server → Message DB (persist).
ClientLoad BalancerServiceQueueCacheDatabase
Clients hold long-lived WebSockets to chat servers; messages route via a queue to the recipient's connection.

Problem

Design a chat system like WhatsApp that delivers messages between users in real time, works across flaky mobile networks, and scales to billions of messages per day.

The hard parts are not storage but connection management (hundreds of millions of concurrent long-lived connections), low-latency fanout to the right device, and correct delivery semantics — sent, delivered, and read receipts — even when the recipient is briefly offline.

Functional Requirements

  • 1:1 and group messaging with text, media, and attachments.
  • Delivery guarantees: message shows sent → delivered → read ticks.
  • Online/last-seen presence and typing indicators.
  • Offline delivery: queue messages until the recipient reconnects.
  • Message ordering preserved within a conversation.

Non-Functional Requirements

  • End-to-end latency under ~200 ms for online recipients.
  • High availability; a server crash must not drop undelivered messages.
  • Horizontally scalable connection tier for hundreds of millions of concurrent sockets.
  • Durable message store; no data loss on ack.
  • Efficient on battery and bandwidth for mobile clients.

Capacity Estimation

Daily active users
~2B users
Messages/day
~100B → ~1.2M msg/s avg, ~5M/s peak
Concurrent connections
~500M open WebSockets
Avg message size
~100 B text; media stored separately
Message write throughput
~1.2M writes/s to message store
Storage/day (metadata)
~100B × 300B ≈ 30 TB/day

APIs

WS connect
connect(authToken) → session bound to chat server
SEND
sendMessage(convId, clientMsgId, body) → {serverMsgId, ts}
ACK
ackDelivery(serverMsgId, status: delivered|read)
GET /messages
history(convId, beforeTs, limit) → Message[]

Components

  • Connection LB: layer-4 load balancer with sticky routing to keep a socket pinned to one chat server.
  • Chat servers: stateful WebSocket handlers; hold the live connection and register userId → server in the session store.
  • Session store (Redis): maps each online user to their chat server + last-seen for presence.
  • Message bus (Kafka): decouples send from deliver; routes messages to the server hosting the recipient.
  • Message DB (Cassandra): durable, write-optimized store partitioned by conversation for history and offline queue.
  • Media service: uploads go to blob storage; only a reference is sent through the chat path.

Request Flow

  1. 1Client opens a WebSocket; the LB pins it to a chat server, which records userId → server in Redis.
  2. 2Sender emits a message; the chat server assigns a serverMsgId, persists it to Cassandra, and returns a sent ack.
  3. 3The server publishes the message to Kafka keyed by recipient.
  4. 4The recipient's chat server consumes it, looks up the live socket, and pushes the message down.
  5. 5Recipient device acks delivered; on view it acks read — receipts route back to the sender the same way.
  6. 6If the recipient is offline, the message stays in the durable store and is replayed on reconnect.

Flow Diagram

Diagram with 5 components: Sender, Chat GW, Msg Bus, Delivery, Recipient. Connections: Sender → Chat GW (1 · send); Chat GW → Msg Bus (2 · enqueue); Msg Bus → Delivery (3 · route); Delivery → Recipient (4 · push).
ClientGatewayQueueService
Send → bus → delivery service pushes to the recipient.

Data / Storage Design

Messages are stored in a wide-column store like Cassandra, partitioned by conversationId and clustered by timestamp so history reads are a single-partition range scan. This gives cheap ordered reads and high write throughput.

The offline queue is just unacked rows in the same table (or a per-user inbox), so there is no separate durable queue to keep consistent. Media never travels through the message pipeline — the client uploads to blob storage and the message carries a URL + content hash. Presence and routing (userId → server) live in Redis with short TTLs since they are ephemeral and rebuildable.

Data Model

Diagram with 4 components: Chat Svc, Redis Session, Conv Part A, Conv Part B. Connections: Chat Svc → Redis Session (1 · session); Chat Svc → Conv Part A (2 · persist); Chat Svc → Conv Part B.
ServiceCacheDatabase
Redis holds live sessions; Cassandra persists by conversation.

Scaling Strategy

The connection tier scales horizontally: add chat servers and the LB spreads new sockets across them. Because state (which user is on which server) lives in Redis and Kafka handles routing, any server can deliver to any user.

Kafka is partitioned by recipient (or conversation) to preserve per-conversation ordering while parallelizing across brokers. The message store is sharded by conversation. Group fanout is done server-side: one send expands to N deliveries via the bus, keeping the client protocol simple.

Bottlenecks & Failure Scenarios

  • Chat server crash: all its sockets drop; clients reconnect and re-register, replaying unacked messages — messages already persisted are safe.
  • Redis session loss: presence is stale; mitigated by TTLs and re-registration on reconnect.
  • Group fanout amplification: a large group multiplies writes/deliveries; cap group size and batch fanout.
  • Hot conversation partition: a viral group skews one shard — split or add a secondary partition key.
  • Duplicate delivery: dedupe with clientMsgId idempotency keys on the client.

Design Tradeoffs

Pros
  • WebSockets give true push with minimal latency and battery cost vs polling.
  • Kafka decouples send from deliver, absorbing spikes and enabling offline delivery.
  • Stateless-ish servers behind Redis routing scale linearly.
Cons
  • Stateful connections complicate deploys and load balancing (sticky sessions).
  • Server-side fanout for huge groups is expensive.
  • Extra hop through the message bus adds a few ms of latency.

Common Interview Questions

How do you deliver a message when the recipient is offline?

It is persisted in the durable store as unacked. On reconnect the client's chat server replays undelivered messages ordered by timestamp, then marks them delivered.

How do you guarantee message ordering?

Partition Kafka and the message store by conversation. All messages for a conversation flow through one partition, giving a total order per conversation without global ordering cost.

Why WebSockets over long polling?

Persistent bidirectional sockets avoid repeated handshakes, cut latency to server-push speeds, and dramatically reduce battery and bandwidth for hundreds of millions of idle-but-connected clients.

60-Second Revision
  • →Persistent WebSockets + sticky LB = real-time push.
  • →Redis maps userId → chat server for routing & presence.
  • →Kafka decouples send from deliver and enables offline queue.
  • →Cassandra partitioned by conversation for ordered history.
  • →Receipts (sent/delivered/read) are just acks flowing back.

Related

Everything on Archprimer is connected — follow the graph.