When the answer is RabbitMQ

Every Kafka interview eventually asks "and when would you use RabbitMQ instead?", and a surprising number of Kafka deployments are answering that question wrongly in production — a task queue with a log's operational weight, or a log rebuilt badly on a broker that was never one. The two are not competitors with a winner; they are different shapes, and this lesson is the shape of the other one.

#A broker, not a log

Kafka is a log: producers append, the broker keeps everything for the retention period, and consumers keep their own position and read at their own pace, as many times as they like. RabbitMQ (and AMQP generally) is a broker: a producer hands a message to an exchange, the exchange routes it into zero or more queues by rules called bindings, a consumer takes it from a queue, and once the consumer acknowledges it the message is gone. The broker's job is delivery, not storage; a queue with no consumers grows, a queue with a consumer empties, and "replay yesterday's messages" is not a question it answers.

producer ──▶ exchange ──(binding: routing key "order.placed")──▶ queue: inventory ──▶ consumer
                       ──(binding: "order.*")───────────────────▶ queue: analytics ──▶ consumer

#Exchanges, bindings and routing keys

The exchange type decides how a message's routing key is matched against bindings:

Routing lives in the broker's configuration, which is the sharpest difference from Kafka: there, a consumer subscribes to a topic and filters in code; here, a queue is bound to the messages it wants and receives nothing else. Adding a consumer for order.placed is declaring a queue and a binding — usually from the consumer's own startup code, idempotently — and the producer never changes.

Queues can be durable (survive a broker restart) and messages persistent (written to disk), and both are needed for a message to survive a crash; the default is neither. Quorum queues, replicated by Raft across nodes, are the current answer for anything that must not be lost; classic mirrored queues are deprecated.

#Acknowledgement, prefetch and the dead-letter exchange

A consumer receives a message and later sends an ack; until it does, the message is unacked and invisible to others, and if the consumer's connection dies the broker requeues it. That is at-least-once delivery, the same guarantee Kafka's manual commit gives, with the same consequence: the consumer must be idempotent. Ack after the work, not before (auto-ack is at-most-once, which is fine for metrics and wrong for orders). Nack or reject with requeue=false sends it to the queue's dead-letter exchange, if one is configured — and it should be, because the alternative for a poison message is a requeue loop that pins a consumer forever. Set x-message-ttl and x-max-length on queues, too: an unbounded queue is the backpressure lesson's unbounded buffer, and a broker whose memory is full stops accepting publishes from everyone.

Prefetch (basic.qos) is how many unacked messages a consumer may hold. It is the throughput-versus-fairness knob: a prefetch of 1 hands each message to whichever consumer is free (fair, slow); a prefetch of 100 lets a fast consumer batch (fast, and one slow message holds 99 behind it). Ten to fifty is the usual range for short tasks; one for long ones.

@RabbitListener(queues = "inventory")
void on(OrderPlaced event, Channel ch, @Header(AmqpHeaders.DELIVERY_TAG) long tag) throws IOException {
    reserve(event);                               // idempotent, keyed on event.id
    ch.basicAck(tag, false);
}

With AcknowledgeMode.AUTO (Spring's default, not the broker's) the container acks after the listener returns and nacks on an exception, and spring.rabbitmq.listener.simple.retry.* adds the retry-with-backoff and the DLQ that the Kafka course's retries lesson built by hand.

#Ordering, throughput and the comparison

A single queue with a single consumer delivers in order. Add a second consumer and ordering is gone — two messages are being processed at once, and a redelivery after a failure lands behind messages that arrived later. Kafka's answer, ordering per key within a partition, has no equivalent here; consistent-hash exchanges (a plugin) approximate it by routing each key to one of several queues, each with one consumer, which is a partition rebuilt by hand.

The honest comparison:

Use RabbitMQ when the messages are work to be done — send this email, resize this image, call this API — and the questions are routing, priority, retry and fairness; the message has no value once handled. Use Kafka when the messages are facts that happened — an order was placed — that several consumers will read, now and later, and the log is worth keeping. A service that emits OrderPlaced to Kafka and dispatches SendConfirmationEmail through RabbitMQ is not confused; it is using each for its shape.