Use when designing caching strategies (cache-aside, write-through, write-behind), implementing distributed locks, building rate limiters, leaderboards, real-time streams (XADD/consumer groups), pub/sub, or tuning eviction policies. Triggers: thundering-herd on cache miss, dogpile on key expiry, Redlock vs SET-NX-PX choice, sliding-window rate limiter, hot-key on a single cluster slot, big-key blowup, MULTI/EXEC across slots, KEYS in production. NOT for Redis Cluster operations/admin (different domain), embedded KV (SQLite, leveldb), in-process LRU caches, or Memcached.
Use when designing caching strategies (cache-aside, write-through, write-behind), implementing distributed locks, building rate limiters, leaderboards, real-time streams (XADD/consumer groups), pub/sub, or tuning eviction policies. Triggers: thundering-herd on cache miss, dogpile on key expiry, Redlock vs SET-NX-PX choice, sliding-window rate limiter, hot-key on a single cluster slot, big-key blowup, MULTI/EXEC across slots, KEYS in production. NOT for Redis Cluster operations/admin (different domain), embedded KV (SQLite, leveldb), in-process LRU caches, or Memcached.
metadata
{"category":"Backend & Infrastructure","tags":["redis","caching","rate-limiting","distributed-locks","streams","sorted-sets"],"provenance":{"kind":"first-party","owners":["port-daddy"]},"pairs-with":[{"skill":"background-job-orchestrator","reason":"Redis Streams + consumer groups are the durable-queue substrate this skill designs; job orchestration semantics (retries, dead-letter, scheduling) live there."},{"skill":"event-driven-architecture-expert","reason":"Choosing pub/sub vs Streams here is one leg of the broader event-delivery-guarantee design that skill owns."},{"skill":"websocket-streaming","reason":"Redis pub/sub is the standard fan-out layer behind multi-node WebSocket/SSE broadcast."}],"io-contract":{"kind":"deliverable","consumes":["[Truncated]","[Truncated]"],"produces":["[Truncated]","[Truncated]"]}}
Redis Patterns Expert
Redis is a data-structure server. The interesting part isn't the commands — it's the patterns. Most of the trouble comes from treating Redis as a database when you should treat it as a coordination primitive, or vice versa.
When to use
Adding a cache layer in front of a slow data source.
Need a distributed lock (single-leader election, mutual exclusion).
Rate limiting an API by user/IP/key.
Real-time leaderboards or sorted feeds.
Event streams (XADD/XREAD with consumer groups) for multi-consumer durable queues.
Pub/sub for notifications (best-effort, single delivery attempt).
Core capabilities
Caching strategies
Cache-aside (most common):
asyncfunctiongetUser(id: string) {
const cached = await redis.get(`user:${id}`);
if (cached) returnJSON.parse(cached);
const fresh = await db.users.findById(id);
// SET with EX in one command — never SET then EXPIRE.await redis.set(`user:${id}`, JSON.stringify(fresh), 'EX', 300 + Math.floor(Math.random() * 60));
return fresh;
}
The + random() jitter prevents the dogpile when many keys expire together. Without it, every cache key set during a deploy expires within the same minute 5 minutes later → thundering herd.
Stale-while-revalidate with a soft + hard TTL:
constSOFT = 300, HARD = 600;
const entry = await redis.get(key);
if (entry) {
const { value, mtime } = JSON.parse(entry);
if (Date.now() - mtime < SOFT * 1000) return value;
// Stale but valid — refresh in background, return cached.voidrefresh(key);
return value;
}
// No entry — block on origin.
Single-flight — collapse concurrent regeneration to one origin call:
const token = crypto.randomUUID();
const acquired = await redis.set(lockKey, token, 'NX', 'PX', leaseMs);
if (!acquired) thrownewError('lock not acquired');
try {
// ... critical section, must complete before leaseMs ...
} finally {
await redis.eval(releaseScript, 1, lockKey, token);
}
The token prevents you from releasing someone else's lock if your work overran the lease. Don't use GET + DEL separately — that's a TOCTOU race.
Redlock (multi-instance) is contentious; for most applications a single Redis primary with a finite lease is fine. If you need stronger guarantees, use a real consensus system (etcd, Consul).
Rate limiting — sliding window with Lua
-- sliding-window.lua-- KEYS[1] = bucket key ARGV[1] = window seconds ARGV[2] = limit ARGV[3] = now-mslocal now = tonumber(ARGV[3])
local window = tonumber(ARGV[1]) * 1000
redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window)
local count = redis.call('ZCARD', KEYS[1])
if count >= tonumber(ARGV[2]) thenreturn0end
redis.call('ZADD', KEYS[1], now, now)
redis.call('PEXPIRE', KEYS[1], window)
return1
Token bucket is cheaper if you can tolerate burstiness; sliding window is fairer.
Sorted sets — leaderboards and time-series
// Top scorersawait redis.zincrby('game:scores', points, userId);
await redis.zrevrange('game:scores', 0, 9, 'WITHSCORES');
// Time-series with score = epochawait redis.zadd('user:42:events', Date.now(), JSON.stringify(event));
await redis.zrangebyscore('user:42:events', Date.now() - 3600_000, '+inf');
// Cap to last 1000 eventsawait redis.zremrangebyrank('user:42:events', 0, -1001);
Streams + consumer groups
Durable queue with multiple consumers and at-least-once delivery:
// Producerawait redis.xadd('jobs', '*', 'type', 'send_email', 'to', user.email);
// One-time setupawait redis.xgroup('CREATE', 'jobs', 'workers', '$', 'MKSTREAM');
// Consumerconst messages = await redis.xreadgroup(
'GROUP', 'workers', `worker-${pid}`,
'COUNT', 10, 'BLOCK', 5000,
'STREAMS', 'jobs', '>',
);
for (const [stream, entries] of messages ?? []) {
for (const [id, fields] of entries) {
try {
awaithandle(fields);
await redis.xack('jobs', 'workers', id);
} catch {
// No XACK → message stays in the PEL (pending entries list).
}
}
}
// Reclaim work from a crashed peerconst pending = await redis.xpending('jobs', 'workers', 'IDLE', 60000, '-', '+', 100);
for (const { id } of pending) {
await redis.xclaim('jobs', 'workers', `worker-${pid}`, 60000, id);
}
// Cap stream growthawait redis.xadd('jobs', 'MAXLEN', '~', 100000, '*', ...fields);
The ~ makes MAXLEN approximate (cheaper). PEL is your story for poison messages — set a max-deliveries threshold and dead-letter beyond it.
At-most-once delivery; subscribers offline at publish time miss the message. For durable jobs, use Streams.
Lua scripts
EVAL ships the script every time; EVALSHA after SCRIPT LOAD is faster. Atomic — the whole script runs without other clients interleaving. KEYS are passed for cluster routing; non-key arguments go in ARGV. Cluster requires all KEYS to live on the same slot — use hash tags {user:42} for related keys.
Eviction policies
maxmemory 100mb
maxmemory-policy allkeys-lru
Policy
When to use
noeviction
Redis as durable store (you should be using a real DB).
allkeys-lru
Pure cache.
allkeys-lfu
Cache where some keys are hot for a long time.
volatile-lru
Mixed: some keys must persist (no TTL), evict only TTL'd ones.
volatile-ttl
Evict shortest-TTL first; useful for staged data.
Anti-patterns
KEYS in production
Symptom: Latency spikes correlate with admin commands.
Diagnosis:KEYS * blocks the server while it scans every key.
Fix:SCAN cursor-based iteration. For Cluster, scan each shard.
DEL on a 10M-item structure
Symptom: Single command takes seconds; replicas lag; clients time out.
Diagnosis:DEL is synchronous in the main thread.
Fix:UNLINK (non-blocking, async free). For sorted sets, batch via ZSCAN + ZREM chunks.
SET then EXPIRE
Symptom: Occasional keys with no TTL.
Diagnosis: Crash between SET and EXPIRE leaves the key forever.
Fix:SET key val EX seconds in one command.
Big-key OOM
Symptom: Single hash >1M fields; replication breaks; OOM on restart.
Diagnosis: Unbounded growth — usually session data or user-item maps.
Fix: Shard by user prefix (session:abc:user:42), or move to a sorted-set + TTL pattern.
Pub/sub for durable jobs
Symptom: Workers restart, miss the work, customer support tickets.
Diagnosis: Pub/sub is at-most-once. Subscribers offline at publish lose the message.
Fix: Use Streams + consumer groups. Always.
Hot-key on a single cluster slot
Symptom: One node CPU at 100%, others idle.
Diagnosis: Cache key collision → all reads on one slot.
Fix: Add a per-key salt or shard the key (user:42:cache:0, …:1) and randomize reads.
Quality gates
No KEYS in production code (verify by linter or grep).
Every cache key has TTL with jitter (5-15% randomization).
Distributed locks use ownership tokens; release is via Lua script.
Stream consumers handle XPENDING reclaim from crashed peers.
Streams capped with MAXLEN ~ N.
maxmemory and maxmemory-policy set explicitly per-environment.
Big collections sharded by some key prefix; no unbounded growth.
Pub/sub used only for ephemeral notifications, never durable jobs.
UNLINK over DEL for any structure with thousands of items.
Deterministic Audit
Before committing to a Redis design (or reviewing another agent's), write it as a JSON
plan matching schemas/redis-patterns-plan.schema.json and run the deterministic auditor:
auditRedisPatterns(plan) (in scripts/redis_patterns_audit.mjs) turns this skill's
anti-patterns and Quality Gates into machine-checkable rules over structured fields —
no keyword matching: pub/sub chosen for a durable queue, a cache key with no TTL or no
jitter, SET then EXPIRE instead of one atomic command, a lock without an ownership
token or with a non-Lua release, a Streams consumer with no PEL reclaim, KEYS in
production, blocking DEL on a big key, and noeviction backing a pure cache. It
returns { pass, score, findings, recommendations } so a reviewer or CI gate can reject
a plan without re-deriving the reasoning. examples/sample-input.json is a
correctly-jittered cache-aside plan (pass: true). Version history lives in
CHANGELOG.md.
NOT for
Redis Cluster admin/operations — failover, slot migration, persistence tuning is a separate skill (no dedicated skill yet).
Webhook receiver dedup keyed in Redis — Redis is fragile for idempotency. → webhook-receiver-design (use a DB unique constraint instead).
Embedded KV stores (SQLite, leveldb, RocksDB) — different consistency model.
In-process LRU (lru-cache npm) — single-node, no shared state.
Memcached — fewer data structures, simpler semantics.
DragonflyDB / KeyDB — Redis-compatible but with different operational characteristics.
