Kafka vs RabbitMQ
Kafka is a durable, replayable distributed log for streaming; RabbitMQ is a flexible message broker with rich routing and per-message acks. Log vs broker.
By Ratish Jain · Updated
The Decision
Both move messages between services, but their core abstractions differ. Kafka is a distributed, append-only commit log: messages persist on disk, are partitioned for parallelism, and consumers track their own offset so they can replay history at will.
RabbitMQ is a traditional message broker built around exchanges, bindings and queues. It excels at flexible routing and per-message acknowledgement, typically deleting a message once it is consumed.
So it depends on whether you need a durable stream you can reprocess (Kafka) or smart routing with fine-grained delivery guarantees and low-latency task dispatch (RabbitMQ).
Side by Side
| Kafka | RabbitMQ | |
|---|---|---|
| Model | Distributed commit log | Broker with exchanges + queues |
| Consumption | Pull by offset; message stays | Push then ack; message removed |
| Replay | Rewind to any retained offset | No native replay once acked |
| Routing | Topic + partition key only | Direct, topic, fanout, headers |
| Ordering | Guaranteed per partition | Per queue, weaker under scaling |
| Throughput | Very high, millions/sec | High, but lower ceiling |
| Best fit | Streaming, event sourcing, logs | Task queues, RPC, complex routing |
Kafka Strengths & Weaknesses
- Retains and replays messages, enabling reprocessing and multiple independent consumers
- Massive throughput via partitioned, sequential disk writes
- Natural fit for event sourcing, stream processing and audit logs
- No rich routing; consumers must filter and manage offsets
- Heavier to operate (brokers, partitions, rebalancing, ZooKeeper/KRaft)
- Overkill for simple point-to-point task queues
Decision Guide
Choose Kafka for high-throughput event streams, log aggregation, event sourcing, or when multiple consumers must independently read and replay the same data. Choose RabbitMQ for task/work queues, request-reply RPC, and cases needing complex routing rules, priorities, or precise per-message acknowledgement and dead-lettering.
Real-World Examples
- Created Kafka to unify activity streams and metrics pipelines
- Uber
- Uses Kafka for trip events and real-time streaming analytics
- Reddit / Instacart
- Use RabbitMQ (or Celery on it) for background task processing
- Netflix
- Runs Kafka fleets for event ingestion at scale
Interview Questions
Each consumer group tracks its own offset independently, so multiple groups read the same partitions at their own pace, and messages remain until retention expires — not deleted on read like RabbitMQ.
When you need flexible routing (fanout/topic/headers), per-message acks with requeue and dead-letter queues, priorities, or low-latency RPC-style dispatch — routing and delivery semantics Kafka does not natively offer.
- →Kafka = replayable partitioned log; consumers pull by offset.
- →RabbitMQ = broker; push-ack, message deleted after consumption.
- →Kafka wins on throughput, retention, and replay.
- →RabbitMQ wins on routing flexibility and fine-grained delivery.
- →Kafka for streams/event sourcing; RabbitMQ for task queues/RPC.
Related
Everything on Archprimer is connected — follow the graph.