Code10x
Redis fundamentalsmedium3-5 years

A work queue built on `LPUSH` + `BRPOP` loses in-flight items whenever a worker crashes mid-job, because a popped list item is already gone from Redis the moment it's popped. A Redis stream with a consumer group is proposed instead. What does the pending entries list actually guarantee, and what obligation does it put on the workers?

A plain list's BRPOP removes and returns an item in one step — once a worker has it, Redis has no further record of it, so a worker that crashes before finishing loses that item permanently with nothing left to retry. A stream with a consumer group instead tracks, for every entry handed out via XREADGROUP, whether that entry has been acknowledged (XACK) yet — the pending entries list is exactly that record, and an entry stays in it, still recoverable, until it's acknowledged. Another worker can then claim it (XAUTOCLAIM) after it's been pending too long, which is at-least-once delivery: the tradeoff is that the same entry can genuinely be processed twice (once by the worker that crashed after finishing but before acknowledging, once by whoever claims it), so consumers have to be written to handle being run on the same entry more than once without corrupting anything.

The lesson behind it →
PreviousA sliding-window rate limiter — allow 10 requests per rolling second per user — is implemented by calling `ZREMRANGEBYSCORE` to trim old entries, then `ZCARD` to count what's left, then conditionally `ZADD`, as three separate Redis commands from application code. Under concurrent requests from the same user, more than 10 get allowed in some windows. What's wrong, and how does wrapping it in a Lua script fix it?Next A home page built from a 300 ms query is cached for a minute. When its entry expires, 200 concurrent requests all miss the cache at once and all run the expensive query. Walk through why a per-instance single-flight lock only partly fixes this across four application instances, and why a Redis-based rebuild lock introduced a second bug on top of the first fix.
More on Redis fundamentals
A service increments a page-view counter in Redis by doing `GET counter`, adding one in application code, then `SET counter <newValue>`. Under load from several instances, the final count is consistently lower than the number of requests. What's happening, and what's the one-line fix?A sliding-window rate limiter — allow 10 requests per rolling second per user — is implemented by calling `ZREMRANGEBYSCORE` to trim old entries, then `ZCARD` to count what's left, then conditionally `ZADD`, as three separate Redis commands from application code. Under concurrent requests from the same user, more than 10 get allowed in some windows. What's wrong, and how does wrapping it in a Lua script fix it?