| name | caching-strategies |
| description | Load when adding or reviewing any cache — Redis/Memcached, CDN, HTTP caching, in-process memoization, or materialized views — or when debugging stale data, cache stampedes, low hit ratios, or invalidation bugs. Also load when someone proposes "just cache it" as a performance fix. |
Caching Strategies
Compact checklist. The standard machinery — cache-aside, delete-don't-set (and its residual read-old/fill-late race), single-flight locks, XFetch, serve-stale-while-revalidate, TTL jitter, negative-cache sentinels, key versioning/generation counters, replica-lag double-delete, fail-open-with-breaker, Vary/private hygiene — is assumed known; it appears here only as anchors so nothing gets skipped under pressure.
Before caching anything
- Fix the query first: a 900ms query an index makes 5ms needs no cache, no invalidation, no stampede surface.
- Write down the maximum staleness the business tolerates (not what engineering finds convenient) and who is hurt when the bet loses. No answer = latent correctness bug with good latency.
- Never cache: uncommitted reads; anything used to enforce an invariant (balance/inventory gates — the source of truth decides); secrets in shared caches.
- Auth/permissions/entitlements: TTL ≤ 60s or don't cache — staleness there is a security incident.
Invalidation strategy by data type (choose by failure mode, not hit ratio)
| Data | Strategy |
|---|
| Rarely changes, staleness cheap | TTL (minutes–hours) |
| Changes on write paths you own | Delete-on-write + TTL backstop |
| Written by many services/writers | CDC/event-driven invalidation + TTL backstop |
| Derived aggregates (counts, feeds) | Short TTL or periodic rebuild — never per-write invalidation of a hot aggregate |
| Writers you can't enumerate | TTL only — incomplete explicit invalidation is worse than none |
TTL fails predictably (bounded, self-healing); event-driven fails unpredictably (missed event = stale forever). Every explicit-invalidation scheme gets a TTL safety net. Batch jobs, admin tools, and second services are the write paths that skip your delete hook — that's what the backstop and CDC are for.
Hot-key checklist (apply all)
- TTL jitter (
base * uniform(0.9, 1.1)) — keys populated together expire together.
- Single-flight per key (in-process lock; cross-process
SET key:lock <token> NX EX with delete-if-token-matches); losers serve stale.
- Serve-stale: physical TTL ~2× logical expiry so stale data exists to serve during recompute and origin outages — but decide per key class; stale prices/permissions may be worse than down.
- XFetch for the hottest keys.
- Read-rate hot keys (celebrity problem): one key at 100k req/s lives on ONE Redis shard — adding shards does nothing. Fixes: in-process L1 with 1–5s TTL in front of Redis, or key replication (
key#{rand(0,9)} read fanout, write/invalidate all copies). This is the piece most designs miss: sharding solves keyspace load, never single-key load.
Key design
- Key = full function signature: entity, tenant, locale, currency, flag-state, schema version. Run the "second tenant / second locale / second flag-state" test.
- Schema version in the key from day one (
price:v7:{id}:{ccy}): class-wide invalidation becomes a one-line version bump at deploy; without it you're running SCAN+UNLINK over 200M keys racing new bad writes.
- Per-entity generation counter for unbounded derived keys:
user:{id}:gen embedded in dependents; invalidate-all = one INCR.
- Normalize multi-valued inputs (sort params) before keying; hash long keys, log the preimage.
Pitfall one-liners (audit every one on review)
- Writer SETs cache instead of DELETE → old value can win the race and persist to TTL.
- Cache fills from a lagging read replica right after delete-on-write → fresh-looking stale fill (delayed double-delete, LSN-gated fill, or primary fills).
- Errors/timeouts cached through the success path → self-inflicted per-key outage.
- Negative cache using
null-that-means-miss → no-op; sentinel must be distinct, short TTL, deleted on the create path.
Cache-Control: public or missing Vary on authenticated responses → cross-user leak; Vary: Cookie → hit ratio silently ~0.- Deploy flushes all in-process caches simultaneously → synchronized stampede on shared tier.
- Unbounded local dict /
functools.lru_cache on instance methods (holds self, cache is per-class, no TTL) → memory leak; use cachetools.TTLCache or cached_property.
- Write-behind acknowledging writes it can lose → cache became the (non-durable) source of truth.
- No Redis client timeout/breaker → cache outage becomes hard-dependency outage; and verify the origin survives 0% hit ratio or pair fail-open with load shedding.
- Global 95% hit ratio hiding a 20% ratio on the expensive namespace → measure per key-class, weighted by origin cost saved; watch eviction rate and staleness-at-serve alongside.
Verification / self-check
- Per cached value, one line each: max tolerable staleness? every write path? what deletes/expires on each path? what happens at 0% hit ratio? Any "unclear" = design not done.
- Interleave a writer and reader at every step: can an older value be written into the cache after a newer one?
- Every entry has a TTL; every cache has a size bound; hot keys have single-flight AND jitter; miss-heavy attacker-reachable lookups have negative caching.
- HTTP/CDN: read
Cache-Control and Vary off a real response, not the config's intent.
Delta notes (vs Opus 4.8 baseline, audited 2026-07)
- Probed 13 claims: 13 baseline (cut/compressed), 0 partial, 0 delta.
- Opus cold reproduced everything load-bearing: delete-vs-set race, stampede arithmetic + full mitigation stack incl. XFetch formula, version/generation invalidation, replica-lag double-delete, fail-open + origin-capacity question, negative-cache sentinels, lru_cache leak, Vary/private hygiene.
- Skill retained as a compact review checklist; only marginal expansions kept: celebrity-key single-shard fanout (L1/key-replication) and the business-staleness framing.
