Not everything should be cached. Cache data that is read frequently, expensive to compute, tolerant of brief staleness, and unlikely to change between reads. Caching mutable, low-read data adds complexity without benefit.
Decision test: If the data is read 10x more than it is written and a few seconds of staleness is acceptable, it is a strong caching candidate.
2. Choose the Right Caching Pattern
Pattern
How it works
Best for
Cache-aside
App checks cache, on miss reads DB, writes to cache
General purpose; simple, widely understood
Read-through
Cache itself loads from DB on miss
Frameworks with cache-provider abstraction
Write-through
Writes go to cache and DB synchronously
Strong consistency requirements
Write-behind
Writes go to cache, async flush to DB
Write-heavy workloads; eventual consistency OK
Refresh-ahead
Cache proactively refreshes before expiry
Predictable access patterns; low-latency reads
Default recommendation: Start with cache-aside. It is the simplest, most portable, and gives the application full control over caching behavior.
Database -- Source of truth; materialized views and query caching reduce load
5. Invalidation is Harder Than Caching
"There are only two hard things in Computer Science: cache invalidation and naming things." Start with TTL-based expiration and layer on event-driven invalidation for critical data paths. Never rely solely on manual invalidation.
6. Agentic Workflow Considerations
When an AI agent is implementing or reasoning about caching:
Identify the consistency requirement first -- Ask whether the consumer tolerates stale data and for how long before choosing a strategy.
Prefer explicit TTLs over unbounded caching -- Every cache entry should have an expiration. Zombie entries from missing invalidation logic are a common production incident.
Consider cache warming on deploy -- Cold caches after deployment cause latency spikes. Pre-populate critical entries.
Log cache hit/miss ratios -- If hit ratio drops below 80%, the cache may be poorly sized or the key space too large.
Test invalidation paths -- Cache bugs are often invisible until stale data causes user-facing issues. Include invalidation in integration tests.
When caching LLM responses -- Evaluate whether prompt caching (provider-level, exact prefix match) or semantic caching (application-level, similarity match) is appropriate. Prompt caching is cheaper and simpler; semantic caching handles paraphrased queries.
Workflow
Identify -- Determine what data is a caching candidate (high read-to-write ratio, expensive to compute, tolerant of staleness).
Select layer -- Choose the appropriate cache layer(s): HTTP, CDN, in-memory, distributed, or database.
Select pattern -- Pick a caching pattern (cache-aside, read-through, write-through, write-behind) based on consistency needs.
Configure eviction -- Set eviction policy (LRU/LFU), max memory, and TTL values.
Implement invalidation -- Define how and when cached data is invalidated (TTL, event-driven, tag-based, versioned keys).
Instrument -- Add metrics for cache hit/miss ratio, latency, eviction rate, and memory usage.
Test -- Verify cache behavior under load, test invalidation correctness, and simulate cache failures.
Monitor -- Track hit ratios in production; tune TTLs and cache sizes based on observed patterns.
Quick Reference
Cache-Aside Pattern (Most Common)
asyncfunctiongetUser(userId: string): Promise<User> {
// 1. Check cacheconst cached = await redis.get(`user:${userId}`);
if (cached) returnJSON.parse(cached);
// 2. Cache miss -- read from databaseconst user = await db.users.findById(userId);
if (!user) thrownewNotFoundError('User not found');
// 3. Populate cache with TTLawait redis.set(`user:${userId}`, JSON.stringify(user), 'EX', 300); // 5 min TTLreturn user;
}
asyncfunctionupdateUser(userId: string, data: Partial<User>): Promise<User> {
// 1. Update database (source of truth)const user = await db.users.update(userId, data);
// 2. Invalidate cacheawait redis.del(`user:${userId}`);
return user;
}
Caching pattern selected and documented (cache-aside, read-through, write-through, write-behind)
TTL set on every cache entry (no unbounded entries)
Eviction policy configured (LRU/LFU with max memory limit)
Cache invalidation strategy implemented and tested
Cache key design is deterministic, namespaced, and under 512 bytes
Cache stampede prevention in place (locks, jitter, or stale-while-revalidate)
Graceful degradation on cache failure (falls through to source of truth)
Cache hit/miss ratio instrumented and alerted (target > 80% hit ratio)
HTTP Cache-Control headers set appropriately for each response type
Static assets served with immutable + long max-age + fingerprinted filenames
Sensitive data excluded from caching (no-store on PII, auth tokens)
Cache warming strategy for cold starts / deployments considered
Serialization format chosen (JSON for debuggability, MessagePack/Protobuf for performance)
Load tested with cache cold, warm, and under invalidation pressure
When to Escalate
Distributed consistency requirements -- If the system requires strong consistency across cache nodes (not just eventual), consult with the architecture team. Distributed cache coherence protocols add significant complexity.
Cache infrastructure sizing -- When Redis memory exceeds available RAM, data partitioning (clustering) and eviction tuning require infrastructure expertise. Estimate working set size before provisioning.
Multi-region cache replication -- Cross-region cache replication introduces latency tradeoffs and conflict resolution challenges. Requires infrastructure and networking review.
Cache-related data incidents -- If stale cached data causes user-facing data integrity issues (e.g., showing another user's data, stale financial data), treat as a severity-1 incident and involve the platform team.
LLM caching cost optimization -- When prompt caching savings plateau and semantic caching is under consideration, evaluate embedding model costs, vector DB infrastructure, and cache hit quality thresholds with the ML platform team.
Framework-level caching abstractions -- If the framework provides built-in caching (e.g., Next.js ISR, Rails fragment caching), prefer framework conventions over custom implementations and consult framework documentation.
