Storagemedium3-5 years

What's the difference between a topic with `cleanup.policy=delete` and one with `cleanup.policy=compact`, and when would you use compaction?

delete (the default) expires whole segments once they're older than retention.ms or the topic exceeds retention.bytes — it's a time or size window, and once a record ages out it's gone regardless of whether anyone consumed it. compact is different in kind: it keeps the latest record for every key forever, deleting only the older, superseded records with the same key in the background. That turns a log into something closer to a table — the current state of every entity, replayable from the start to rebuild a cache or a database — which is what delete can never give you, since it deletes by age with no regard for whether a key's latest value is still in the retained window.

The lesson behind it →