WhatsApp / Chat
A real-time messaging system built on persistent WebSocket connections, with message fanout, delivery/read receipts, and online presence at billions-of-messages-per-day scale.
By Ratish Jain · Updated
Problem
Design a chat system like WhatsApp that delivers messages between users in real time, works across flaky mobile networks, and scales to billions of messages per day.
The hard parts are not storage but connection management (hundreds of millions of concurrent long-lived connections), low-latency fanout to the right device, and correct delivery semantics — sent, delivered, and read receipts — even when the recipient is briefly offline.
Functional Requirements
- 1:1 and group messaging with text, media, and attachments.
- Delivery guarantees: message shows sent → delivered → read ticks.
- Online/last-seen presence and typing indicators.
- Offline delivery: queue messages until the recipient reconnects.
- Message ordering preserved within a conversation.
Non-Functional Requirements
- End-to-end latency under ~200 ms for online recipients.
- High availability; a server crash must not drop undelivered messages.
- Horizontally scalable connection tier for hundreds of millions of concurrent sockets.
- Durable message store; no data loss on ack.
- Efficient on battery and bandwidth for mobile clients.
Capacity Estimation
- Daily active users
- ~2B users
- Messages/day
- ~100B → ~1.2M msg/s avg, ~5M/s peak
- Concurrent connections
- ~500M open WebSockets
- Avg message size
- ~100 B text; media stored separately
- Message write throughput
- ~1.2M writes/s to message store
- Storage/day (metadata)
- ~100B × 300B ≈ 30 TB/day
APIs
- WS connect
- connect(authToken) → session bound to chat server
- SEND
- sendMessage(convId, clientMsgId, body) → {serverMsgId, ts}
- ACK
- ackDelivery(serverMsgId, status: delivered|read)
- GET /messages
- history(convId, beforeTs, limit) → Message[]
Components
- Connection LB: layer-4 load balancer with sticky routing to keep a socket pinned to one chat server.
- Chat servers: stateful WebSocket handlers; hold the live connection and register
userId → serverin the session store. - Session store (Redis): maps each online user to their chat server + last-seen for presence.
- Message bus (Kafka): decouples send from deliver; routes messages to the server hosting the recipient.
- Message DB (Cassandra): durable, write-optimized store partitioned by conversation for history and offline queue.
- Media service: uploads go to blob storage; only a reference is sent through the chat path.
Request Flow
- 1Client opens a WebSocket; the LB pins it to a chat server, which records
userId → serverin Redis. - 2Sender emits a message; the chat server assigns a
serverMsgId, persists it to Cassandra, and returns a sent ack. - 3The server publishes the message to Kafka keyed by recipient.
- 4The recipient's chat server consumes it, looks up the live socket, and pushes the message down.
- 5Recipient device acks delivered; on view it acks read — receipts route back to the sender the same way.
- 6If the recipient is offline, the message stays in the durable store and is replayed on reconnect.
Flow Diagram
Data / Storage Design
Messages are stored in a wide-column store like Cassandra, partitioned by conversationId and clustered by timestamp so history reads are a single-partition range scan. This gives cheap ordered reads and high write throughput.
The offline queue is just unacked rows in the same table (or a per-user inbox), so there is no separate durable queue to keep consistent. Media never travels through the message pipeline — the client uploads to blob storage and the message carries a URL + content hash. Presence and routing (userId → server) live in Redis with short TTLs since they are ephemeral and rebuildable.
Data Model
Scaling Strategy
The connection tier scales horizontally: add chat servers and the LB spreads new sockets across them. Because state (which user is on which server) lives in Redis and Kafka handles routing, any server can deliver to any user.
Kafka is partitioned by recipient (or conversation) to preserve per-conversation ordering while parallelizing across brokers. The message store is sharded by conversation. Group fanout is done server-side: one send expands to N deliveries via the bus, keeping the client protocol simple.
Bottlenecks & Failure Scenarios
- Chat server crash: all its sockets drop; clients reconnect and re-register, replaying unacked messages — messages already persisted are safe.
- Redis session loss: presence is stale; mitigated by TTLs and re-registration on reconnect.
- Group fanout amplification: a large group multiplies writes/deliveries; cap group size and batch fanout.
- Hot conversation partition: a viral group skews one shard — split or add a secondary partition key.
- Duplicate delivery: dedupe with
clientMsgIdidempotency keys on the client.
Design Tradeoffs
- WebSockets give true push with minimal latency and battery cost vs polling.
- Kafka decouples send from deliver, absorbing spikes and enabling offline delivery.
- Stateless-ish servers behind Redis routing scale linearly.
- Stateful connections complicate deploys and load balancing (sticky sessions).
- Server-side fanout for huge groups is expensive.
- Extra hop through the message bus adds a few ms of latency.
Common Interview Questions
It is persisted in the durable store as unacked. On reconnect the client's chat server replays undelivered messages ordered by timestamp, then marks them delivered.
Partition Kafka and the message store by conversation. All messages for a conversation flow through one partition, giving a total order per conversation without global ordering cost.
Persistent bidirectional sockets avoid repeated handshakes, cut latency to server-push speeds, and dramatically reduce battery and bandwidth for hundreds of millions of idle-but-connected clients.
- →Persistent WebSockets + sticky LB = real-time push.
- →Redis maps userId → chat server for routing & presence.
- →Kafka decouples send from deliver and enables offline queue.
- →Cassandra partitioned by conversation for ordered history.
- →Receipts (sent/delivered/read) are just acks flowing back.
Related
Everything on Archprimer is connected — follow the graph.