Collectionshard8+ years

A rates cache is `rates.computeIfAbsent(ccy, c -> rateClient.fetch(c))` on a `ConcurrentHashMap`. When the provider's 30 ms call becomes a 12-second call, requests for currencies that were already cached start timing out, but only for some currencies. Explain exactly which calls block and which do not, and fix it without adding a library.

computeIfAbsent keeps its exactly-once guarantee by holding the lock of the hash bin the key lives in while your function runs. An insert into a populated bin never skips that lock, so while one thread is 12 seconds inside fetch for one currency, every other write or compute on a key in the same bin waits for it. Keys in other bins are unaffected, which is why the failure looks data-dependent: it is a hashing coincidence. The cure is to stop doing the slow work inside the function: store a CompletableFuture as the value, so the bin is held for the microsecond it takes to create the future and the waiting happens outside the map.

The lesson behind it →
More on Collections