Archprimer
TradeoffIntermediate 8 min read

Kafka vs RabbitMQ

Kafka is a durable, replayable distributed log for streaming; RabbitMQ is a flexible message broker with rich routing and per-message acks. Log vs broker.

messagingkafkarabbitmqstreaming

By Ratish Jain · Updated

Diagram with 5 components: Producer, Kafka, RabbitMQ, Consumer Group, Workers. Connections: Producer → Kafka (append); Producer → RabbitMQ (publish); Kafka → Consumer Group (poll offset); RabbitMQ → Workers (push+ack).
ServiceQueue
A producer feeding a replayable log versus a routing broker.

The Decision

Both move messages between services, but their core abstractions differ. Kafka is a distributed, append-only commit log: messages persist on disk, are partitioned for parallelism, and consumers track their own offset so they can replay history at will.

RabbitMQ is a traditional message broker built around exchanges, bindings and queues. It excels at flexible routing and per-message acknowledgement, typically deleting a message once it is consumed.

So it depends on whether you need a durable stream you can reprocess (Kafka) or smart routing with fine-grained delivery guarantees and low-latency task dispatch (RabbitMQ).

Side by Side

KafkaRabbitMQ
ModelDistributed commit logBroker with exchanges + queues
ConsumptionPull by offset; message staysPush then ack; message removed
ReplayRewind to any retained offsetNo native replay once acked
RoutingTopic + partition key onlyDirect, topic, fanout, headers
OrderingGuaranteed per partitionPer queue, weaker under scaling
ThroughputVery high, millions/secHigh, but lower ceiling
Best fitStreaming, event sourcing, logsTask queues, RPC, complex routing

Kafka Strengths & Weaknesses

Pros
  • Retains and replays messages, enabling reprocessing and multiple independent consumers
  • Massive throughput via partitioned, sequential disk writes
  • Natural fit for event sourcing, stream processing and audit logs
Cons
  • No rich routing; consumers must filter and manage offsets
  • Heavier to operate (brokers, partitions, rebalancing, ZooKeeper/KRaft)
  • Overkill for simple point-to-point task queues

Decision Guide

Decision Guide

Choose Kafka for high-throughput event streams, log aggregation, event sourcing, or when multiple consumers must independently read and replay the same data. Choose RabbitMQ for task/work queues, request-reply RPC, and cases needing complex routing rules, priorities, or precise per-message acknowledgement and dead-lettering.

Real-World Examples

LinkedIn
Created Kafka to unify activity streams and metrics pipelines
Uber
Uses Kafka for trip events and real-time streaming analytics
Reddit / Instacart
Use RabbitMQ (or Celery on it) for background task processing
Netflix
Runs Kafka fleets for event ingestion at scale

Interview Questions

How does Kafka let two teams consume the same data without interfering?

Each consumer group tracks its own offset independently, so multiple groups read the same partitions at their own pace, and messages remain until retention expires — not deleted on read like RabbitMQ.

When is RabbitMQ the better choice despite Kafka's popularity?

When you need flexible routing (fanout/topic/headers), per-message acks with requeue and dead-letter queues, priorities, or low-latency RPC-style dispatch — routing and delivery semantics Kafka does not natively offer.

60-Second Revision
  • →Kafka = replayable partitioned log; consumers pull by offset.
  • →RabbitMQ = broker; push-ack, message deleted after consumption.
  • →Kafka wins on throughput, retention, and replay.
  • →RabbitMQ wins on routing flexibility and fine-grained delivery.
  • →Kafka for streams/event sourcing; RabbitMQ for task queues/RPC.

Related

Everything on Archprimer is connected — follow the graph.