Caching patterns
A 2.6 ms query, a 75 µs Redis GET and a 5 ns local map; cache-aside and its relatives, local against distributed, TTLs, LFU beating LRU at one tenth of the data, and negative caching.
A cache is a copy of data kept somewhere faster than where the data lives. That sentence contains both halves of every caching decision: faster, which is why you add one, and copy, which is the cost — a copy can be missing, too big for the space you gave it, or different from the original.
This lesson measures the first half and names the second. The experiments ran against Redis 7.4 in one container and a Java client in another; the "database" is a simulated 2 ms query, so that the numbers isolate the cache.
What a cache buys
Three ways to fetch the same product:
database query (simulated) 2,592,324 ns
Redis GET (another container) 74,609 ns
local map get 5 ns- A database query of about 2.6 ms — and in production, often much more under load.
- A Redis GET over the network: about 75 microseconds, roughly 35 times faster than the query, because it is a hash lookup in memory with a network round trip in front of it.
- A local, in-process map: about 5 nanoseconds once the JIT has compiled the lookup — fifteen thousand times faster than Redis, because there is no network at all.
Speed is only one of the things bought. A cache also absorbs load the database would otherwise take: every cache hit is a query that did not happen. For many systems that is the real reason — the database could answer fast enough, but not ten thousand times a second.
The four patterns
The patterns differ in who talks to the cache and when it is written:
| pattern | read | write | the catch |
|---|---|---|---|
| cache-aside | the application checks the cache; on a miss, it reads the database and fills the cache | the application writes the database, then deletes or updates the cache entry | the application owns the logic, and the race between a read-fill and a write-delete |
| read-through | the application asks the cache; the cache loads from the database on a miss | as cache-aside | the cache needs a loader — common in local caches like Caffeine, rare in Redis |
| write-through | as read-through | the application writes to the cache, which writes the database synchronously | the cache is now on the write path; its failure is a write failure |
| write-behind | as read-through | the cache acknowledges, and writes the database later, in batches | writes acknowledged and not yet durable can be lost |
Cache-aside is the default for a service in front of Redis: it keeps the database as the only source of truth, and a cache outage degrades to slower reads rather than to failed writes. The code is short:
Product get(long id) {
Product cached = cache.get(id);
if (cached != null) return cached;
Product fresh = repository.findById(id);
cache.put(id, fresh, Duration.ofMinutes(5));
return fresh;
}
void update(Product p) {
repository.save(p);
cache.evict(p.id());
}In Spring, the same shape is declarative — reference code, not run in these experiments:
@Cacheable(cacheNames = "products", key = "#id")
public Product get(long id) { return repository.findById(id).orElseThrow(); }
@CacheEvict(cacheNames = "products", key = "#p.id")
public void update(Product p) { repository.save(p); }@Cacheable works through a proxy, so a call from inside the same class bypasses the cache — the same self-invocation trap as @Transactional. And the two lines in update are not atomic: the stampedes and consistency lesson measures how often that leaves the cache wrong.
Local, distributed, or both
The 15,000× gap between a local map and Redis is tempting. The trade:
| local (in-process: Caffeine, a map) | distributed (Redis) | |
|---|---|---|
| hit latency | nanoseconds | tens to hundreds of microseconds |
| shared between instances | no — each instance has its own copy | yes |
| survives a deploy | no | yes |
| memory | the service's own heap | a separate server |
| invalidation | per instance; others keep the old value | one delete reaches everyone |
The critical row is the fourth. With ten instances each holding a local copy, an update invalidates the copy on one instance. The other nine serve the old value until their entries expire, and a user whose requests are balanced across instances sees the value change back and forth. That incoherence is acceptable for data that is allowed to be a few seconds old — configuration, reference data, feature flags — and not for anything a user just edited.
A common compromise is two levels: a small local cache with a very short TTL in front of Redis, for the hottest keys only. It removes most network round trips while bounding how long instances can disagree to that short TTL.
TTL: how long a copy may be wrong
Every cache entry should have a time-to-live, even when the application also invalidates on write. The TTL is the upper bound on how long any bug, race or missed invalidation can serve wrong data. Choosing it:
- From the data's tolerance for staleness, not from what "feels right". A product description can be an hour old. A price at checkout cannot be cached at all.
- With jitter. Entries written together — a warm-up at startup, a batch job — expire together, and the database takes all the misses at the same moment. Adding a random 10% to each TTL spreads that out.
- Shorter than the invalidation you do not trust. If an update path might forget to evict, the TTL is the fix that works anyway.
Eviction: when the cache is full
A cache has a memory limit, and when it is reached something must go. Redis supports several maxmemory-policy settings, and the choice matters in proportion to how uneven the traffic is.
An experiment: 100,000 products of about 1 KB each — 100 MB of potential cache — in a Redis limited to 12 MB, so roughly one product in ten fits. Two hundred thousand requests with realistic skew (a Zipf distribution, where a few products get most of the traffic), cache-aside on every request:
allkeys-lru keys held 10,507 hit ratio 59.5% evicted 70,441 database queries 80,971
allkeys-lfu keys held 10,507 hit ratio 62.4% evicted 64,757 database queries 75,287
allkeys-random keys held 10,507 hit ratio 56.4% evicted 76,753 database queries 87,283- LRU (least recently used) evicts what has not been touched lately.
- LFU (least frequently used) evicts what is touched rarely, using a small decaying counter per key.
- random evicts anything.
With one tenth of the data in memory, the cache answered 56–62% of requests — that is what skewed traffic gives you, and it is why caches work at all. LFU was best here, because with a stable popularity distribution, frequency is a better predictor than recency: an unpopular product requested once is recently used, but not frequently. LFU beat LRU by about three points of hit ratio — 5,684 fewer database queries out of 200,000.
Two cautions about the policy names. volatile-* policies only evict keys with a TTL, and noeviction — the default — makes writes fail when memory is full. A cache configured with noeviction stops accepting new entries instead of making room, which usually shows up as errors from SET.
Negative caching: remembering what does not exist
Cache-aside caches values. A request for something that does not exist finds nothing in the cache, queries the database, finds nothing there either, and caches nothing — so the next request for it does the same. A bug that requests a deleted product in a loop, or a scraper walking through ids that do not exist, goes straight through the cache to the database every time. This is called cache penetration.
Twenty thousand requests, one in ten for one of 50 product ids that do not exist:
negative caching off: 3,019 database queries for 20,000 requests
negative caching on : 1,050 database queries for 20,000 requestsWithout it, every one of the 2,000 requests for a missing product reached the database. With a sentinel value cached for 30 seconds — "we looked, it is not there" — each missing id cost one query, and the database load fell by two thirds.
The sentinel needs a short TTL, because the thing that did not exist may be created: a product added a moment after a failed lookup should not stay invisible for an hour. For open-ended id spaces, where an attacker can request a different missing id every time, a negative cache alone does not help; a Bloom filter of ids that do exist, checked before the cache, rejects most of them without touching Redis or the database.