Caching strategies

• Cache-aside (lazy loading) — the application checks the cache first; on a miss, it reads from the source of truth, populates the cache, and returns the value. Simple and the most common pattern; the cache can go stale if the source changes without an explicit invalidation, and every miss pays full source latency.
• Read-through — the cache itself (or a layer directly in front of it) owns fetching from the source on a miss, so the application only ever talks to the cache. Cleaner application code, same staleness characteristics as cache-aside.
• Write-through — every write goes to the cache and synchronously to the source before the write is acknowledged. Keeps the cache always consistent with the source, at the cost of every write paying the source’s write latency in addition to the cache write.
• Write-behind (write-back) — writes go to the cache immediately and are asynchronously flushed to the source later, batched or delayed. Lowest write latency, but a cache failure before the flush completes can lose writes, and readers of the source-of-truth directly (bypassing the cache) see stale data until the flush happens.

Choice depends on read/write ratio and how much staleness or write-loss risk is acceptable — cache- aside is the default for read-heavy workloads tolerant of brief staleness; write-through/write-behind matter more when the cache is closer to being the primary write path.

TTL-based expiry is simple but forces a trade-off between staleness (long TTL) and cache-miss rate / source load (short TTL) that does not adapt to how often the underlying data actually changes. Explicit invalidation (delete or update the cache entry when the source changes) is more precise but introduces real distributed-systems hazards:

• Missed invalidations — any write path that forgets to invalidate (a bulk update, a direct-to-database script, a second service writing the same data) leaves a stale entry with no natural expiry to eventually correct it if there is no TTL as a backstop.
• Invalidation race — a write updates the source and invalidates the cache, but a concurrent read that started just before the write re-populates the cache with the now-stale pre-write value right after the invalidation — a genuine ordering race, not a rare edge case at real concurrency.
• Fan-out invalidation — one underlying change can invalidate many derived cache entries (a changed user profile invalidating every cached page that embedded it), and finding all of them requires either a dependency-tracking scheme or accepting a coarser invalidation (clear a whole namespace) that gives up some cache efficiency for correctness.

Practical systems combine both: TTL as a correctness backstop, explicit invalidation for responsiveness, and are honest that perfect immediate consistency between cache and source under concurrent writes is genuinely hard, not solved by picking either mechanism alone.

When a hot cache key expires (or the cache is cold-started), every concurrent request for that key simultaneously misses and hits the source at once — a sudden spike that can overload the source exactly when it is least prepared for it, sometimes cascading into an outage that then keeps failing to repopulate the cache because the source is now too overloaded to answer even one request. Mitigations:

• Lock / single-flight — the first request to miss acquires a lock and fetches from the source; concurrent requests for the same key wait on that fetch (or serve a slightly stale value) instead of each independently hitting the source.
• Early / probabilistic expiration — recompute the value slightly before the TTL actually expires, with a small random jitter per key so many keys with the same nominal TTL do not all refresh at the exact same moment.
• Stale-while-revalidate — serve the expired-but-still-cached value immediately while asynchronously refreshing it in the background, so no request pays the full source latency at the moment of expiry.
• Randomized TTL jitter at write time — avoid setting many keys to expire at literally the same timestamp in the first place (e.g. after a bulk cache warm).

• Data that must always be current (real-time balances, live inventory near zero) — a stale cache hit is a correctness bug, not a performance trade-off, and the business cost of being wrong can exceed the cost saved by caching.
• Low hit rate, high-cardinality keys — caching something almost never requested twice adds memory pressure and cache-management overhead for near-zero benefit; check actual hit rate before assuming a cache helps.
• Masking a real underlying performance problem — caching over a slow query can hide the need to actually fix the query/index, and the moment the cache is cold (deploy, eviction, a new key) the underlying slowness resurfaces unmitigated and unexpectedly.
• Adding a new failure mode and a new consistency problem for a workload that was already fast enough without it — every cache is one more thing that can be stale, one more thing that can fail, and one more piece of operational surface area; it should be added because a measured problem needs it, not by default.

Size from the working set — the volume of data actually accessed repeatedly within a timeframe that matters, not the total size of everything that could theoretically be cached; a cache far smaller than the working set thrashes (evicts entries that would soon be reused), while one much larger than the working set wastes memory with no further hit-rate gain past that point — plot hit rate vs. cache size on real traffic to find the actual knee of the curve rather than guessing. Eviction policy: LRU (evict least-recently-used) is the default and works well when recent access predicts future access, which is true for most workloads; LFU (evict least-frequently-used) suits workloads where some keys are consistently hot regardless of recency (LRU can evict a very-frequently -but-not-most-recently used key in favor of a one-off recent access); TTL-based eviction is appropriate when correctness/staleness bounds matter more than hit-rate optimization. Many production caches (Redis with allkeys-lru, CDN edge caches) default to LRU-family policies precisely because they are a robust default across a wide range of access patterns without needing workload-specific tuning.

Each layer caches a different thing for a different reason, and they compose rather than substitute for each other:

• Client-side (browser/app) — avoids a network round-trip entirely for genuinely static or rarely-changing data; the cheapest cache hit possible, but staleness is hardest to control (invalidating a client cache requires the client to check back in).
• CDN / edge — caches responses close to the user, mainly for content that is the same across many users (static assets, public API responses); reduces both latency and origin load at scale.
• Application-level (Redis/Memcached) — caches computed or fetched data specific to your service’s own logic (session data, expensive query results, rendered fragments); most flexible, closest to your invalidation logic.
• Database query/buffer cache — caches at the lowest level, mostly out of your application’s direct control, absorbing repeated identical queries and hot pages.

A well-designed system deliberately decides what belongs at each layer based on how shareable (same value for many callers) and how volatile the data is — highly shareable, rarely-changing data pushes toward CDN/client; per-user or fast-changing data pushes toward application-level caching with tighter invalidation, or no caching at all.