Intermediate · INTERACTIVE LESSON

Cache stampede and request coalescing

Compare duplicate origin work with per-key single-flight under the same arrivals.

The interactive player is loading. If it does not appear, you can still read the explanation below.

THE IDEA BEHIND THE STEPS

How it works

Both policies receive the same bounded request schedule and start with identical caches. Keys arrive in total bursts, at floor(request index / burst size) × interval seconds, and are assigned round-robin to clients. Burst size 1 spaces arrivals; choose an interval longer than origin time to avoid overlapping calls. Clients can have multiple outstanding requests.

Without coalescing, every cache miss starts a separate origin fetch. With coalescing, the first miss atomically reserves an in-flight slot for that key in one application process. Later misses for the same key wait for that flight. Different keys can fetch independently. On completion, all attached requests receive the same success or failure, and the slot is removed. Failed fetches do not fill the cache; a later miss can try again.

Origin fetches take a fixed positive number of simulated seconds and fail for the configured keys. Completions happen before same-time arrivals; completion ties use policy order (uncoalesced then coalesced), then flight ID. Arrivals use request ID then policy order. A successful completion fills the cache for the fixed TTL. Warm keys expire at TTL seconds; all expiry is inclusive, t >= expiresAt. Reads do not refresh TTL. Cache hits complete immediately. Values are assumed unchanged; network animation adds no simulated time.

Origin fetches count started calls, joins count misses attached to an existing flight, and success/failure counts terminal logical requests. Waiting clients are shown by request ID because a client may issue several requests. Fewer origin calls here are not measured throughput or latency improvements.

Single-flight trades duplicate origin work for shared waiting and shared failure. This model assumes atomic lookup/reservation within one process; it is not a distributed lock. Separate application instances would coordinate independently unless a separate distributed protocol were designed. Coalescing guarantees neither global deduplication nor availability. No cancellation, retries, hung origin, process crashes, capacity eviction, writes or distributed races are modeled. All started fetches eventually finish in this bounded simulation; production systems need timeout and cleanup policies.