Partitionseasy0-2 years

A topic has 6 partitions and a consumer group has 20 instances. Why are 14 of them doing nothing, and why can't you fix it by adding more consumers?

Within a consumer group, each partition is owned by exactly one consumer at a time — that's how Kafka guarantees a record is processed once by the group without two members racing on it. A topic with 6 partitions can therefore have at most 6 members doing work; the other 14 are connected, heartbeating, and passing every health check, but they own nothing and process nothing. Adding a 21st consumer changes nothing either — the ceiling is the partition count, not the consumer count, and the only way to raise it is to raise the partition count.

The lesson behind it →