What Is a Cache Stampede?

Quick answer
A cache stampede is many concurrent requests for the same expired key all missing the cache and recomputing it at the same time.
It is a synchronisation problem, not a capacity problem: the work is duplicated rather than merely increased.
The classic symptoms are a periodic latency spike aligned to the time-to-live and a database load graph with narrow, regular peaks.
The standard fixes are single-flight or lock-based recomputation, jittered expiry times, and serving stale values while one request refreshes in the background.
Adding more capacity helps briefly and does not fix it, because the duplicated work scales with concurrency, not with traffic.

What is a cache stampede?

It is what happens when a cached value that many requests depend on expires and every request that arrives afterwards tries to rebuild it. Each request sees a miss, each independently computes the same expensive value, and the origin receives the full concurrency of the site aimed at one operation. The cache was supposed to prevent exactly that, and for the duration of the rebuild it is worse than no cache, because the requests are also all racing on the same key.

Two conditions create it: a key that is both popular and expensive to compute, and expiry timing that is not staggered. If traffic is low, or the value is cheap, a stampede is invisible. If the value takes seconds to compute and is requested thousands of times a second, the rebuild load can exceed what the origin can serve, which turns a cache expiry into an outage.

Recognising it in graphs

The signature is temporal alignment. Latency spikes appear at regular intervals matching the time-to-live rather than correlating with traffic, and the database shows narrow peaks of the same shape. Because all instances share the same key and often the same expiry duration, they expire together and miss together, which is why the pattern looks like a heartbeat rather than a ramp.

The second signature is a mismatch between request rate and work rate. Query counts to the database jump to roughly the number of concurrent requests for the key, while total traffic is unchanged. Comparing cache misses per key against request volume per key makes it obvious: one key accounts for a large share of misses despite being cached for its entire lifetime otherwise.

The three fixes, and when each applies

Single-flight, sometimes called request coalescing, ensures only one request computes a missing value while the others wait for that result. It converts N concurrent computations into one, and the others receive the same value. The implementation is a lock or an atomic set-if-not-exists on the key: the winner computes, the losers either wait briefly or fall back. It is the most direct fix and the one to reach for when the value must be fresh.

Jittered expiry staggers the time-to-live so keys do not expire in lockstep across instances or across a set of related keys. Adding a random fraction to each lifetime removes the alignment that makes the spikes periodic. Serving stale data while refreshing — the pattern browsers call stale-while-revalidate — keeps answering from the old value and refreshes in the background, so a user never waits for the rebuild at all.

# Single-flight rebuild: one winner, everyone else gets the result.
def get_or_refresh(cache, key, ttl, compute, lock_ttl=10):
    hit = cache.get(key)
    if hit is not None:
        return hit
    # Only one caller acquires the lock; the rest fall through gracefully.
    if cache.set(f'lock:{key}', '1', nx=True, ex=lock_ttl):
        try:
            value = compute()
            cache.set(key, value, ex=ttl)
            return value
        finally:
            cache.delete(f'lock:{key}')
    # Someone else is rebuilding: serve a slightly stale copy if we have one.
    stale = cache.get(f'stale:{key}')
    if stale is not None:
        return stale
    # No stale copy: wait briefly and try once more rather than hammering.
    import time; time.sleep(0.05)
    return cache.get(key) or compute()

What does not work

Raising the time-to-live reduces the frequency of the spikes but not their size, and it lengthens the staleness window. Adding database capacity absorbs one more stampede and fails at the next concurrency level, because the duplicated work scales with the number of simultaneous misses rather than with the traffic. Taking the cache out entirely removes the stampede and replaces it with the constant load the cache was introduced to avoid.

A common half-fix is to warm the cache on deploy or on a schedule without staggering. That moves the spike rather than removing it, and a prewarm of many keys at once can be worse than the expiry it replaced. Prewarming works when it is staggered, rate-limited, and treats a failed key as non-fatal, which is a different discipline from caching.

Preventing it before you ship the cache

Decide, for each cached value, whether a stale answer is acceptable. If it is, use stale-while-revalidate and never make a user wait for a rebuild. If it is not, use single-flight so the expensive computation happens once. In both cases, add jitter to the lifetime so keys do not align. Then load-test with an expired cache, which is the test that actually reproduces the condition — testing with a warm cache proves nothing about what happens at expiry.

Where this bites vibecoders
A generated caching layer is usually a decorator with a fixed lifetime and no coordination, which is precisely the stampede configuration: every instance expires the same key at the same time and every request rebuilds it. Because the bug only appears at expiry and under concurrency, a local test with one user never shows it. Ask whether the cached value can be computed twice concurrently and whether that is safe and affordable; if not, it needs coalescing.

Where AI coding assistants get this wrong

Using a fixed time-to-live on a popular key with no coordination between requests.
Assuming a cache prevents load spikes, when an expiry can cause one.
Prewarming many keys at once on a schedule, creating a self-inflicted spike.
Caching an expensive aggregate without any single-flight protection.
Testing performance with a warm cache only.
Serving stale data without marking it, so correctness-sensitive reads are affected.

Checklist

Identify the popular, expensive keys and their time-to-live values.
Decide whether a stale value is acceptable for each one.
Add single-flight rebuild or stale-while-revalidate accordingly.
Add jitter to all lifetimes so keys do not expire together.
Load-test with a cold cache, not a warm one.
Monitor cache misses per key to catch the alignment if it returns.

FAQ

How is a stampede different from a cold cache?

A cold cache is everything missing after a restart, which is a large but one-off load. A stampede is a single key missing for many concurrent requests at once, and it recurs on every expiry. The fixes overlap, but the recurring nature is what makes a stampede a reliability problem rather than a start-up cost.

Does adding a lock solve it completely?

It solves the duplicated computation for that key, provided the lock has a short expiry so a crashed holder does not wedge the key forever, and provided the waiting requests have a bounded wait. A lock without a timeout can turn a stampede into a stall, which is not obviously better. Add a stale fallback for the waiters.

What is the simplest prevention?

Jittered expiry plus stale-while-revalidate. Jitter removes the alignment that makes the spikes periodic, and serving a slightly stale value while one request refreshes means no user ever waits for the rebuild. That combination is a few lines and covers most cases without any locking at all.

Should I cache the value at all if it is this dangerous?

Usually yes, because the alternative is paying the full computation cost on every request. The danger is in the unmanaged expiry, not the caching. A cache with jitter, a stale fallback, and one coalesced refresh is strictly better than no cache for anything expensive enough to be worth caching.