# Chapter 4 — Caching with Redis By the end of this chapter you will: - Implement cache-aside correctly - Invalidate without serving stale forever - Use the right TTL for each access pattern - Avoid the thundering-herd problem --- ## The why > **Your database is the library archive in the basement. Redis is the notepad on the librarian's desk — small, fast, and answers 90% of questions without a trip downstairs.** Walk into a university library. You ask the librarian, "Where's the book on distributed systems?" The librarian doesn't walk into the archive stacks. They flip through a card catalog on their desk (Redis) and point you to Aisle 7, Shelf 3. If the book isn't in the catalog, they walk to the basement (PostgreSQL), find the book, and — here's the key — write a new card for the catalog before handing you the book. Next person asking gets the 2-second answer, not the 200-second one. Caching is the art of keeping frequently-needed data close to the code that needs it. Redis, an in-memory data structure store, is the industry standard for this. It operates at microsecond latency (vs millisecond for PostgreSQL) because it never touches disk for reads. But caching introduces the hardest problem in computer science: **cache invalidation** — when the basement book moves, who updates the card? And the second hardest: **naming things** — what key pattern will survive 6 months of feature development? This chapter covers the four cache strategies (cache-aside, write-through, write-behind, read-through), practical invalidation techniques, cache stampede protection, Redis data structures beyond key-value, and patterns from companies that serve billions of cached requests daily. --- ## Core concepts ### Cache-Aside: The Workhorse Pattern Cache-aside (also called \"lazy loading\") is the most common pattern. The application manages the cache explicitly: ``` Client → App Server → Redis (cache) ↘ PostgreSQL (source of truth) ``` **Read path:** Check Redis → if hit, return cached data → if miss, query PostgreSQL, store in Redis, return data. **Write path:** Write to PostgreSQL → invalidate (delete) the Redis key → next read repopulates from fresh data. ```typescript title="services/user-service-cached.ts" import { Redis } from 'ioredis'; import { Pool } from 'pg'; const redis = new Redis(process.env.REDIS_URL!); const db = new Pool({ connectionString: process.env.DATABASE_URL }); interface UserProfile { id: string; name: string; email: string; avatarUrl: string; } const CACHE_TTL = 3600; // 1 hour export async function getUserProfile(userId: string): Promise { const cacheKey = `user:profile:${userId}`; // 1. Try cache const cached = await redis.get(cacheKey); if (cached) { return JSON.parse(cached) as UserProfile; } // 2. Cache miss — query database const result = await db.query( 'SELECT id, name, email, avatar_url FROM users WHERE id = $1', [userId] ); if (result.rows.length === 0) { throw new Error('User not found'); } const profile: UserProfile = { id: result.rows[0].id, name: result.rows[0].name, email: result.rows[0].email, avatarUrl: result.rows[0].avatar_url, }; // 3. Populate cache (fire-and-forget — don't block the response) redis.setex(cacheKey, CACHE_TTL, JSON.stringify(profile)).catch(err => { console.error('Redis set failed (non-critical):', err.message); }); return profile; } export async function updateUserProfile(userId: string, updates: Partial): Promise { // 1. Write to database (source of truth) await db.query( 'UPDATE users SET name = COALESCE($2, name), avatar_url = COALESCE($3, avatar_url) WHERE id = $1', [userId, updates.name, updates.avatarUrl] ); // 2. Invalidate cache — next read will repopulate with fresh data const cacheKey = `user:profile:${userId}`; await redis.del(cacheKey); } ``` You might be tempted to `redis.setex(cacheKey, TTL, newData)` in `updateUserProfile()`. Don't. If two writes happen simultaneously, you can end up with stale data in Redis (write A updates DB, write B updates DB, write B updates Redis, write A updates Redis — Redis now has A's old data). Deleting the key is safe: the next read always sees the latest database state. ### Cache Stampede Protection A cache stampede (or \"thundering herd\") happens when a popular key expires and 100 simultaneous requests all hit the database at once, each trying to recompute the same value. ``` Key "trending:posts" expires at T=0 T=1ms: 100 requests all get cache miss T=2ms: 100 requests all hit PostgreSQL simultaneously T=50ms: PostgreSQL melts under 100 identical queries ``` **Solution: Probabilistic Early Recomposition (PER).** Before a key expires, probabilistically recompute it early. The probability increases as the key gets closer to expiry: ```typescript title="services/cache-with-stampede-protection.ts" async function getWithPER(key: string, ttl: number, fetcher: () => Promise): Promise { // Check cache const cached = await redis.get(key); if (cached) { const ttlRemaining = await redis.ttl(key); // If TTL < 20% of original, probabilistically recompute early if (ttlRemaining > 0 && ttlRemaining < ttl * 0.2) { const shouldRecompute = Math.random() < 0.3; // 30% chance if (shouldRecompute) { // Fire-and-forget recomputation — don't block current request recomputeInBackground(key, ttl, fetcher); } } return JSON.parse(cached); } // Cache miss — use a lock to prevent stampede const lockKey = `lock:${key}`; const lockAcquired = await redis.set(lockKey, '1', 'NX', 'PX', 5000); // 5s lock if (lockAcquired) { try { // Check cache again — another request might have populated it const recheck = await redis.get(key); if (recheck) return JSON.parse(recheck); // Fetch, cache, return const data = await fetcher(); await redis.setex(key, ttl, JSON.stringify(data)); return data; } finally { await redis.del(lockKey); } } else { // Another request is computing — poll cache with backoff for (let i = 0; i < 10; i++) { await new Promise(r => setTimeout(r, 100 * Math.pow(1.5, i))); const retry = await redis.get(key); if (retry) return JSON.parse(retry); } // Fallback: compute anyway (lock might be stale) const data = await fetcher(); return data; } } async function recomputeInBackground(key: string, ttl: number, fetcher: () => Promise) { try { const data = await fetcher(); await redis.setex(key, ttl, JSON.stringify(data)); } catch (err) { console.error(`Background recompute failed for ${key}:`, err); } } ``` ### Write Strategies: Cache-Aside vs Write-Through vs Write-Behind | Strategy | Write sequence | Pros | Cons | Use case | |---|---|---|---|---| | **Cache-aside** | Update DB → Delete cache key | Simple, safe, cache is always a subset of DB | Next read is always a cache miss (cold) | User profiles, blog posts, product pages | | **Write-through** | Write to cache → Write to DB (sync) | Cache always consistent with DB | Write latency = cache write + DB write | Configuration data, feature flags | | **Write-behind** | Write to cache → Async write to DB | Fastest writes (cache latency only) | Can lose data if Redis crashes before DB flush | High-frequency counters, analytics, rate limiting | | **Read-through** | App reads cache → Cache reads DB on miss | App code never touches DB directly | Adds complexity; cache must know DB schema | When you want DB abstraction via cache layer | **Write-through implementation:** ```typescript title="write-through-example.ts" async function updateConfig(key: string, value: string): Promise { // Write to cache first await redis.set(`config:${key}`, value); // Then write to database await db.query( 'INSERT INTO configs (key, value) VALUES ($1, $2) ON CONFLICT (key) DO UPDATE SET value = $2', [key, value] ); // If db write fails, delete from cache to stay consistent // (A more robust implementation would use a transaction or saga pattern) } ``` ### Cache Invalidation Strategies Phil Karlton famously said: \"There are only two hard things in Computer Science: cache invalidation and naming things.\" Here are your weapons: **1. TTL (Time-To-Live) — the simplest, most robust.** ```typescript redis.setex('trending:posts', 300, JSON.stringify(posts)); // 5 minutes ``` TTL works because it's automatic. Even if your invalidation logic has bugs, the cache self-corrects when TTL expires. Always set a TTL. **2. Explicit invalidation — precise, but fragile.** ```typescript // After updating post #42, delete all related cache keys await redis.del('post:42'); await redis.del('post:42:comments'); await redis.del('user:posts:sharma'); // Author's post list ``` The problem: you must remember every cache key that contains the stale data. Miss one, and users see old data until TTL kicks in. **3. Cache-keys-by-tag — best of both worlds.** ```typescript // When caching, tag the key const postKey = 'post:42'; await redis.setex(postKey, 3600, JSON.stringify(post)); await redis.sadd(`tag:post:42`, postKey); // Add to tag set // When invalidating, delete all keys with that tag const taggedKeys = await redis.smembers(`tag:post:42`); if (taggedKeys.length > 0) { await redis.del(...taggedKeys); await redis.del(`tag:post:42`); } ``` **4. Namespace-based invalidation — the nuclear option.** ```typescript // Use a version number in the key const version = await redis.get('cache:version:v2') || '0'; const key = `v${version}:post:42`; // To invalidate EVERYTHING in namespace v2 await redis.incr('cache:version:v2'); // Old keys become orphaned; TTL eventually cleans them up ``` ### Redis Data Structures Beyond Key-Value Redis is not just a key-value store. Its data structures solve caching problems elegantly: **Sorted Sets for Leaderboards:** ```typescript // Add score await redis.zadd('leaderboard:weekly', user.score, user.id); // Top 10 const top10 = await redis.zrevrange('leaderboard:weekly', 0, 9, 'WITHSCORES'); // User's rank const rank = await redis.zrevrank('leaderboard:weekly', userId); ``` A sorted set with 1M members returns top-10 in microseconds — try that with PostgreSQL `ORDER BY score DESC LIMIT 10` on a table with 10M rows. **Hashes for Partial Updates:** ```typescript // Store user session as hash (each field individually accessible) await redis.hset(`session:${sessionId}`, { userId: 'user_42', ip: '192.168.1.1', lastAccess: Date.now(), cartItemCount: '3', }); // Read only one field const cartCount = await redis.hget(`session:${sessionId}`, 'cartItemCount'); // Increment atomically await redis.hincrby(`session:${sessionId}`, 'cartItemCount', 1); ``` **Lists for Activity Feeds:** ```typescript // Push new activity (trim to last 100) await redis.lpush(`feed:${userId}`, JSON.stringify(activity)); await redis.ltrim(`feed:${userId}`, 0, 99); // Get recent 20 const feed = await redis.lrange(`feed:${userId}`, 0, 19); ``` ### Memoization: Function-Level Caching Memoization caches the return value of a function for given arguments. It's cache-aside at the function level: ```typescript title="lib/memoize.ts" function memoize Promise>( fn: T, options: { ttl: number; keyPrefix: string } ): T { return (async (...args: any[]) => { const cacheKey = `${options.keyPrefix}:${JSON.stringify(args)}`; const cached = await redis.get(cacheKey); if (cached) return JSON.parse(cached); const result = await fn(...args); await redis.setex(cacheKey, options.ttl, JSON.stringify(result)); return result; }) as T; } // Usage const getOrderTotal = memoize( async (orderId: string) => { return db.query('SELECT SUM(price * quantity) FROM order_items WHERE order_id = $1', [orderId]); }, { ttl: 300, keyPrefix: 'memo:orderTotal' } ); ``` Expensive computations with limited input variations. `getOrderTotal` is perfect — each `orderId` is computed once, cached for 5 minutes. But `searchProducts(query)` is terrible — the input space is nearly infinite and cache hit rate will be near zero. --- ## Real-world example: How Twitter caches tweet timelines Twitter's timeline rendering is one of the most cache-intensive operations on the internet. When you open Twitter, you see a personalized timeline of tweets from accounts you follow. Computing this from scratch would require: fetch list of followed accounts (hundreds), fetch their recent tweets (thousands), rank by relevance/recency, hydrate with media URLs, like counts, retweet counts — per request. Impossible at Twitter's scale (~500M tweets/day). **Twitter's caching architecture:** 1. **Timeline cache (Redis cluster).** Each user's timeline is pre-computed and stored as a Redis list. When User A (with 10M followers) tweets, Twitter's fanout service pushes the tweet ID into the timeline cache of every follower — a \"write-through\" to cache. When you open Twitter, your timeline is a single `LRANGE` call to Redis. Sub-millisecond. 2. **Tweet cache (separate Redis cluster).** Tweets themselves (text, media URLs, metrics) are cached in a hash per tweet ID. Timeline rendering grabs tweet IDs from the timeline cache, then bulk-loads tweet data from the tweet cache. This decouples timeline composition from tweet content. 3. **Multi-tier caching.** Hot tweets (viral, celebrity) sit in an L1 in-memory cache (memcached/Redis). Warm tweets sit in L2 (flash-backed Redis or SSD). Cold tweets are fetched from Manhattan (Twitter's distributed key-value store, their source of truth). 4. **Cache warming for celebrities.** When a celebrity with 50M followers tweets, pushing to all follower timelines would take minutes. Twitter uses a \"hybrid fanout\": for users with < 10K followers, fanout to all followers' caches. For celebrities, the tweet is NOT pushed — it's merged at read time from the celebrity's tweet list. This is a \"cache vs compute\" tradeoff: it's cheaper to merge at read time for a few celebrities than to push to 50M caches. **Key lesson:** Twitter doesn't have ONE cache. They have specialized caches for timelines, tweets, user profiles, trends, and search — each with its own TTL, eviction policy, and data structure (lists for timelines, hashes for tweets, sorted sets for trends). --- ## Common mistakes | Mistake | Why it's wrong | What to do instead | |---|---|---| | No TTL on cache keys | Stale data lives forever; Redis memory fills with data nobody reads | Always set `EX`/`EXPIRE`; choose TTL based on how often data changes | | Caching without invalidation on writes | User updates profile → sees old data for hours | On write: delete or update all cache keys containing that data | | Single global TTL for everything | User profile (changes weekly) and stock price (changes per second) shouldn't have the same 1-hour TTL | Tune TTL per entity: 5s for stock prices, 1h for profiles, 24h for static config | | Storing large JSON blobs | 1MB cache value → 1ms to deserialize per request → negates Redis speed advantage | Store only needed fields; use Redis hashes for partial access; keep values under 10KB | | Caching before measuring | Adding cache to an endpoint that runs in 5ms and is called 10 times/day | Profile first: cache endpoints with high latency (> 100ms) AND high call volume (> 10 req/s) | | Using Redis as primary database | Redis is not durable by default; AOF/RDB snapshots can lose recent writes | Redis = cache layer (data loss = performance degradation, not business loss). PostgreSQL = source of truth. | | No cache stampede protection | 100 requests hit the database simultaneously when a popular key expires | Use locking (Redis SET NX) or probabilistic early recomputation (PER) | --- ## Production notes - **Redis maxmemory-policy.** Set `maxmemory-policy allkeys-lru` for cache Redis — evict least-recently-used keys when memory fills. Never use `noeviction` for a cache (it will reject writes when full). For session Redis, use `volatile-lru` so non-expiring keys aren't evicted. - **Separate Redis instances.** Use different Redis instances for cache vs sessions vs queues. They have different persistence, eviction, and memory requirements. Don't let a cache eviction policy delete your active jobs. - **Connection pooling.** Don't create a new Redis connection per request. Use `ioredis` with a connection pool or a singleton. Connection creation is expensive (TCP handshake + AUTH). - **Monitor hit rate.** Track `cache_hits / (cache_hits + cache_misses)`. A hit rate below 80% means you're caching the wrong things or your TTL is too short. Use Redis `INFO stats` to get `keyspace_hits` and `keyspace_misses`. - **Cache graceful degradation.** If Redis is down, your app should still work — slower, but functional. Wrap Redis calls in try/catch and fall through to the database. --- ## Performance notes - **Pipelining for bulk operations.** 100 `redis.get()` calls = 100 round trips. Use `redis.pipeline()` or `redis.mget()` to batch them into one round trip: `const values = await redis.mget(keys)`. - **Compression for large values.** For cached responses > 10KB, compress with `zlib` before storing: `redis.setex(key, ttl, zlib.deflateSync(JSON.stringify(data)))`. Saves 60-80% memory at the cost of CPU. - **Lua scripting for atomicity.** The cache stampede lock (check + set) requires atomicity. Use a Lua script executed via `redis.eval()` instead of two separate commands that could have a race condition between them. - **Avoid hotkeys.** If a single Redis key gets 10,000 requests/second, it becomes a bottleneck on one Redis shard. For hot data, replicate across multiple keys (`post:42:replica:1`, `post:42:replica:2`) and randomly pick one at read time. - **Benchmark with realistic payloads.** `redis-benchmark` with `SET key value` (3-byte value) gives unrealistic numbers. Benchmark with your actual payload sizes and data structures for accurate capacity planning. --- ## Security notes - **Redis password is mandatory.** Never deploy Redis without `requirepass`. An exposed Redis on the public internet can be exploited in seconds (attackers run `FLUSHALL`, or worse, write SSH keys via `CONFIG SET dir`). - **No public internet exposure.** Redis should be in a private VPC, accessible only to your application servers. Use TLS (`rediss://`) if traffic leaves the VPC. - **Rename dangerous commands.** In `redis.conf`: `rename-command FLUSHALL ""`, `rename-command CONFIG ""`, `rename-command DEBUG ""`. These commands have no place in a production cache. - **Cache key injection.** If your cache key includes user input (`user:${req.query.id}`), a malicious user can supply `../../admin:secret` as the ID to read other users' cached data. Sanitize keys or hash user input. - **Don't cache authentication tokens.** JWT tokens, API keys, and session secrets should never be in Redis. If Redis is compromised, cached auth data becomes an instant account takeover vector. --- ## Exercises ### Beginner 1. **Add cache-aside to a user profile endpoint.** Take an Express `GET /users/:id` endpoint that queries PostgreSQL. Add Redis caching: check Redis first, on miss query DB and populate Redis with a 1-hour TTL. On `PUT /users/:id`, invalidate the cache key. 2. **Experiment with TTLs.** Cache a \"recent posts\" endpoint with TTLs of 5s, 60s, and 300s. Measure cache hit rate for each under load. Observe how hit rate increases with TTL but data freshness decreases. ### Intermediate 1. **Implement cache stampede protection.** Your \"trending topics\" query takes 2 seconds and is hit 500 times/sec. Add a Redis-based lock (`SET NX`) so only one request computes the result while others wait and poll. Measure the reduction in database load. 2. **Build a leaderboard with sorted sets.** Use Redis sorted sets to implement a weekly leaderboard. Users earn points (simulate with random increments). The API returns top 10, user's rank, and user's score — all from Redis, no database queries. ### Advanced 1. **Multi-tier caching with in-memory + Redis.** Implement a two-tier cache: an in-memory LRU cache (Node.js `Map` with size limit) as L1, Redis as L2. On read: check L1 → check L2 → query DB. On write: update L1 → update L2 → write DB. Measure latency at each tier. Simulate a Redis outage — confirm the app degrades to L1-only, then DB-only. 2. **Cache key namespace versioning.** Implement a versioned cache key strategy where all keys are prefixed with a version number (`v42:user:123`). When a major data migration changes all user data, increment the version in Redis — all old keys become unreachable. Implement a background process that cleans up orphaned keys from old versions. --- ## Interview questions ### Beginner **Q1: What is cache-aside and when would you use it?** **Answer:** Cache-aside means the application manages the cache explicitly: it checks the cache, and on miss, fetches from the database and populates the cache. On writes, it updates the database and invalidates the cache. Use it when read latency matters (user-facing APIs), data is read more often than written, and stale data for a short period is acceptable. **Q2: What is a TTL and why should every cache key have one?** **Answer:** TTL (Time-To-Live) is the duration after which a cache key is automatically deleted by Redis. Every key needs a TTL because: (1) it prevents stale data from living forever if invalidation logic has bugs, (2) it reclaims memory for data that's no longer accessed, (3) it's a safety net — even if you forget to invalidate on write, the data self-corrects at TTL expiry. **Q3: What happens during a cache stampede?** **Answer:** A popular cache key expires. Before any request can repopulate it, 100+ simultaneous requests all experience a cache miss simultaneously. All 100 hit the database with the same expensive query, potentially overloading it. This is also called the \"thundering herd\" problem. Solutions include request locking (only one request computes, others wait) and probabilistic early recomputation (refresh the key before it expires). ### Senior **Q4: When would you use write-through caching instead of cache-aside?** **Answer:** Write-through is appropriate when read-after-write consistency is critical — the application MUST see the latest data immediately after writing. Examples: feature flags (write a flag, next request must reflect it), configuration updates, real-time inventory counts. The tradeoff is write latency: each write takes cache-write + DB-write time instead of just DB-write. I'd also use write-through when the cache is the primary read path and cache misses are expensive enough that I don't want to force one on every write. **Q5: Your Redis hit rate is 95%, but p99 latency is still 200ms. What's happening?** **Answer:** The 5% of cache misses are the problem. If those misses are on expensive queries (complex JOINs, full-text search), their latency dominates the p99. I'd investigate which keys are missing: use Redis `MONITOR` or slowlog to identify the missed keys, then check if those keys have too short a TTL, are never being populated (bug in the caching logic), or are being invalidated too aggressively. I might also implement cache warming — pre-compute expensive keys on a schedule rather than on first request. **Q6: Twitter uses a \"hybrid fanout\" for celebrity tweets. Explain this tradeoff.** **Answer:** For normal users (< 10K followers), when they tweet, Twitter pushes the tweet ID into every follower's timeline cache (write-through fanout). For celebrities (50M+ followers), pushing to all followers would require 50M Redis writes per tweet — that's seconds of write time and enormous memory churn. Instead, Twitter skips the fanout: the celebrity's tweets are stored in their own list. When a follower loads their timeline, Twitter merges tweets from followed accounts' lists on-the-fly. The tradeoff: slightly more read-time computation (merging) in exchange for eliminating millions of cache writes. It's a classic \"compute at write time vs compute at read time\" decision.

Chapter 4 — Caching with Redis

Part of L4