RabbitMQ vs Kafka

  1. What is Rabbit?
  2. What is Kafka?
  3. Why Rabbit MQ?
  4. Why Kafka?
  5. When to use RabbitMQ over Kafka?
  6. When to use Kafka over RabbitMQ?
  7. Challenges of RabbitMQ
  8. Challenges of Kafka

What is Rabbit?

General-purpose message broker, based around message queues, designed with a smart broker / passive consumer model

What is Kafka?

Distributed & partitioned commit log with messaging semantics. Distributed real-time streaming platform.

  • Q? How log a log base system that is real-time streaming
  • Q? log base system are slow but real-time systems are supposed to be fast how does it work

Why Rabiit MQ?

  • Client Libraries
    • Java(4), Spring(3), .Net(6), Ruby(7), Python(4), PHP(7), JavaScript & Node(4), Rust(2), Object-C Scala(1), Other JVM(11), C & C++(4), Go(3)
  • Plugins
    • Tier 1(19) Community: Routing, Auth, Mgmt, Clustering, Logging, Queues, Protocols
  • Message Dash board
  • Powerful routing capabilites it has
  • Easy to Scale by adding/removing competing consumers on a single queue
  • Can be configured for consistency, high availabilty, low latency, high throughput
  • Cluster rolling upgrade via feature flags
  • Easy to get started
  • Supports strict ordering
  • Wider use case: event driven microservices, RPC, ETL (with SCDF), enterprise message bus, pub-sub message, real-time analytics (with Reactor and RabbitMQ Reactive API).

Why Kafka?

  • Very high throughput with data guarantees
  • Massive Scale (It can scale massively)
  • Ability to replay events (It is a log-based system)
  • De-facto standard for streaming platform
  • Ability to plug-n-play consumer groups on a topic
  • Support strict ordering [with a partition]
  • Replace complex data architectures
  • Broad ecosystem
  • Wider use cases: pub-sub messaging, events driven microservices, logs store, streaming, event sourcing, CDC, enterprise data pipelines.

When to use RabbitMQ over Kafka?

If you don’t have specific Kafka requirements, then RabbitMQ gives greater flexibility, can meet high throughput and real-time event-processing needs and has lower cost of operations

  • for example) event replay
  • operational costs there’s both the hardware as well as the operational complexity

Evolving application requirements

Decoupled producer and consumers (using exchanges)

Consumers independently bring their own queue that binds to exchanges

Consuming applications don’t need to process message that aren’t relevant

When to use Kafka over RabbitMQ?

Steaming platform

  • If you need streaming platform If you need have streaming requirements, kafka is usually becoming the choice

Extremely high throughput

  • It can scale it can have parallelism because of the partitions. It can scale really well

Joining multiple streams or streams and tables to enrich the data.

Scale (typically beyond 5 brokers)


  • the individual consumer actually can replay.
  • Traditional message queue where when you read the message the message is gone from queue

Challenges of RabbitMQ

Operational Complexity in resolving network partitions

  • Relative operational complexity is lower than it is with Kafka

Queues are single-threaded

  • Which means that throughput is limited
  • It can’t scale beyond the compute capacity of the broker

Scaling brokers

  • a rabbit cluster it is really optimal to do a three-node cluster. beyond three brokers for example five clusters, you can do that it becomes a little bit complicated. If you scale beyond a five node cluster it really you have some performance impacts.

No events replay

Does not natively support stateful streaming use cases (but can do so with Reactor + RabbitMQ Reactive API with an external store such as Redis)

Challenges of Kafka

No free lunch – operational complexity

Requires a separate Zookeeper cluster

Requires meticulous planning to select partition count

Storage management overheads

Careful coordination needed between teams writing consumer groups and producers

No out of box management & monitoring console

Streaming API support restricted mainly to Java

  • If you want to get the latest versions of the streaming API, Java is the best choice

No out of box solution for upgrade.

  • confluent offer some solution but it is not part of the open-source.

Leave a Reply

Your email address will not be published.