Offsetsmedium0-2 years
A team enabled `enable.auto.commit=true` believing it gives at-least-once delivery. What can it actually lose, and why?
Auto commit doesn't commit "every five seconds" the way it sounds — it commits whatever the previous poll() returned, at the start of the next poll(), once the interval has passed, regardless of whether the consumer actually finished processing that batch. If a batch of 100 records is only half-processed when the interval fires and the next poll commits offset 200, and the process crashes on record 160, records 160–199 are marked done and were never touched — silently lost. That's at-most-once behaviour with a delay, not at-least-once.
PreviousA topic has 6 partitions and a consumer group has 20 instances. Why are 14 of them doing nothing, and why can't you fix it by adding more consumers?Next Replication factor 3, `acks=all`. A team wants writes to survive one broker failing with no data loss and no downtime. What else has to be set, and why does `acks=all` alone not guarantee it?