# Chapter 5 — Caching at Scale By the end of this chapter you will: - Place caches at the right layer for each access pattern - Avoid the stampede - Build a multi-tier cache with TTL hierarchies - Invalidate by tag, not by key --- ## The why > **Caching is the cheat code of performance engineering. A cache hit is 100-1000× faster than a database query. But misuse it, and you'll trade performance problems for correctness problems — and correctness problems are much harder to debug.** You're a researcher who needs facts from a massive encyclopedia. Walking to the library takes 15 minutes. Looking up a fact in the encyclopedia takes 2 minutes. You do this 50 times a day — that's 14 hours of just fetching facts. Then you discover Post-It notes. Every time you look up a fact, you write it on a note and stick it on your monitor. Next time you need that fact, you glance at the Post-It — 2 seconds instead of 17 minutes. This is caching. But problems creep in: - **The encyclopedia updates.** The population of Tokyo changed. Your Post-It is wrong. How do you know to throw it away? (Cache invalidation) - **Your monitor is full.** You have 200 Post-Its but only space for 50. Which ones do you throw away? (Eviction policy) - **Everyone wants the same fact.** 50 researchers all rush to look up the same fact at the same time. The library can't handle 50 simultaneous requests. (Cache stampede) - **You wrote the wrong thing.** Your Post-It says Tokyo has 14 million people, but you misread the encyclopedia (it's 37 million). Now you confidently give wrong answers for weeks. (Cache consistency) Caching at scale is the art of keeping enough Post-Its to make things fast, knowing when to throw them away, and never trusting them more than the source of truth. --- ## Core concepts ### The multi-tier cache architecture Production systems don't have one cache. They have layers — each serving a different purpose with different tradeoffs: ``` Client (Browser) │ ▼ ┌─────────────────────────────────────┐ │ Tier 1: Browser Cache │ <1ms, per-user │ (Cache-Control, ETag, Service Worker)│ └─────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────┐ │ Tier 2: CDN Edge Cache │ 10-50ms, shared per region │ (Cloudflare, CloudFront, Fastly) │ └─────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────┐ │ Tier 3: Application Cache (L1) │ <1ms, per-instance │ (In-memory Map, LRU cache) │ └─────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────┐ │ Tier 4: Distributed Cache (L2) │ 1-5ms, shared across instances │ (Redis, Memcached, Dragonfly) │ └─────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────┐ │ Tier 5: Database │ 5-50ms, source of truth │ (PostgreSQL, MySQL, DynamoDB) │ └─────────────────────────────────────┘ ``` Each tier has a different TTL (Time-To-Live) strategy: - **Browser cache:** Hours to days (static assets), seconds to minutes (API responses) - **CDN:** Minutes to hours (images, CSS, JS), seconds (personalized content) - **App cache (L1):** Seconds to minutes, invalidated by writes - **Redis (L2):** Minutes to hours, shared across all instances - **Database:** Source of truth — no TTL, just data ### Cache invalidation: the hardest problem "There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton Invalidation means removing or updating cached data when the source of truth changes. The strategies, from simplest to most robust: **1. TTL-based (time-to-live):** Cache data for a fixed duration. After the TTL expires, fetch fresh data. Simple but imprecise — stale data lives until the TTL expires. ```typescript // TTL-based: simple, stale until expiry async function getUser(userId: string): Promise { const cacheKey = `user:${userId}`; const cached = await redis.get(cacheKey); if (cached) return JSON.parse(cached); const user = await db.query('SELECT * FROM users WHERE id = $1', [userId]); await redis.set(cacheKey, JSON.stringify(user), { EX: 300 }); // 5 min TTL return user; } // On user update, you have two choices: // (a) Delete the cache key → next read will repopulate // (b) Update the cache key → immediate consistency but risk of overwrite async function updateUser(userId: string, data: Partial): Promise { await db.query('UPDATE users SET ... WHERE id = $1', [userId]); await redis.del(`user:${userId}`); // Invalidate. Simple. Correct. } ``` **2. Write-through:** Update the cache at the same time as the database. Cache is always consistent (within the write transaction). Cost: every write touches the cache, even for data nobody reads. ```typescript // Write-through: cache is always fresh async function updateUserWriteThrough(userId: string, data: Partial): Promise { await db.query('UPDATE users SET ... WHERE id = $1', [userId]); const fresh = await db.query('SELECT * FROM users WHERE id = $1', [userId]); await redis.set(`user:${userId}`, JSON.stringify(fresh.rows[0]), { EX: 3600 }); } ``` **3. Write-behind (write-back):** Write to the cache first, then asynchronously write to the database. Fastest writes, but risk of data loss if cache crashes before DB write completes. Used by high-throughput systems where occasional data loss is acceptable (gaming leaderboards, view counts). **4. Cache-aside (lazy loading):** The most common pattern. Application checks cache first. On miss, loads from DB and populates cache. On write, invalidates cache. Simple, resilient (cache failure doesn't break reads), but suffers from cache stampede on cold starts. **5. Tag-based invalidation:** Instead of tracking individual keys, tag related cache entries. When a user updates their profile, invalidate all entries tagged `user:{userId}`. This handles complex dependencies cleanly. ```typescript // Tag-based invalidation with Redis async function cacheWithTags(key: string, value: any, tags: string[], ttl: number) { await redis.set(key, JSON.stringify(value), { EX: ttl }); for (const tag of tags) { await redis.sadd(`tag:${tag}`, key); } } async function invalidateByTag(tag: string) { const keys = await redis.smembers(`tag:${tag}`); if (keys.length > 0) { await redis.del(...keys, `tag:${tag}`); } } // Usage: await cacheWithTags( `user:${userId}:profile`, profile, [`user:${userId}`, 'profile'], 300 ); // User updates their profile → invalidate everything tagged with their user ID await invalidateByTag(`user:${userId}`); ``` ### The three cache disasters **1. Cache Penetration:** A key that doesn't exist in the database is repeatedly requested. Each request misses the cache, hits the database, finds nothing, and the cycle repeats. An attacker can exploit this by requesting `user:-1`, `user:-2`, etc. — all miss cache, all hit database. ```typescript // Solution: cache the absence (negative caching) async function getUserSafe(userId: string): Promise { const cacheKey = `user:${userId}`; const cached = await redis.get(cacheKey); if (cached === '__NULL__') return null; // Negative cache hit if (cached) return JSON.parse(cached); const user = await db.query('SELECT * FROM users WHERE id = $1', [userId]); if (!user.rows[0]) { // Cache the absence for a short time (prevents DB hammering) await redis.set(cacheKey, '__NULL__', { EX: 60 }); return null; } await redis.set(cacheKey, JSON.stringify(user.rows[0]), { EX: 300 }); return user.rows[0]; } ``` **2. Cache Avalanche:** A large number of cache keys expire simultaneously, causing a flood of requests to the database. This typically happens when many keys share the same TTL. ```typescript // Solution: add jitter to TTLs function ttlWithJitter(baseTTL: number, jitterPercent: number = 0.2): number { const jitter = baseTTL * jitterPercent * Math.random(); return Math.floor(baseTTL - (baseTTL * jitterPercent / 2) + jitter); } // Instead of: EX: 3600 (all expire at exactly 1 hour) await redis.set(key, value, { EX: ttlWithJitter(3600) }); // Expires between 3240-3960s ``` **3. Cache Stampede (Thundering Herd):** A popular cache key expires, and 100 concurrent requests all try to recompute it simultaneously, overwhelming the database. ```typescript // Solution: request coalescing (promise deduplication) const inFlightRequests = new Map>(); async function getOrCompute(key: string, compute: () => Promise, ttl: number): Promise { // Check cache first const cached = await redis.get(key); if (cached) return JSON.parse(cached); // Is someone already computing this key? if (inFlightRequests.has(key)) { return inFlightRequests.get(key)!; // Wait for the in-flight computation } // We're the first. Compute and let others wait. const promise = (async () => { try { const result = await compute(); await redis.set(key, JSON.stringify(result), { EX: ttl }); return result; } finally { inFlightRequests.delete(key); } })(); inFlightRequests.set(key, promise); return promise; } // 100 concurrent requests → 1 database query, 99 wait on the promise const user = await getOrCompute( `user:${userId}`, () => db.query('SELECT * FROM users WHERE id = $1', [userId]), 300 ); ``` ### Redis Cluster: caching at scale When a single Redis instance can't handle the load or data volume, Redis Cluster shards data across multiple nodes: ``` Redis Cluster (6 nodes: 3 primaries + 3 replicas): Hash Slots: 0-16383 ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ Primary A │ │ Primary B │ │ Primary C │ │ Slots 0-5460 │ │Slots 5461- │ │Slots 10923- │ │ │ │ 10922 │ │ 16383 │ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │ │ │ ┌──────┴───────┐ ┌──────┴───────┐ ┌──────┴───────┐ │ Replica A' │ │ Replica B' │ │ Replica C' │ └──────────────┘ └──────────────┘ └──────────────┘ KEY → CRC16(key) % 16384 → hash slot → node ``` Redis Cluster does NOT support multi-key operations that span hash slots. `MGET key1 key2` works only if key1 and key2 hash to the same slot. Use hash tags (`{user:123}:profile` and `{user:123}:settings`) to co-locate related keys on the same slot. Also, Redis Cluster loses writes during network partitions by default — it's an AP system, not CP. ### The hot key problem A single cache key that receives disproportionate traffic creates a hot spot. On a sharded cache, the shard holding that key becomes overloaded while other shards idle. **Solutions:** - **Local (L1) cache:** Cache hot keys in application memory. A 10KB key cached in 100 app instances uses 1MB of RAM but absorbs all hot key traffic. - **Key replication:** Store multiple copies of the hot key with different names (`hotkey:1`, `hotkey:2`, `hotkey:3`). Clients pick a random copy. Each copy lives on a different shard. - **Split the key:** If the hot key is a counter, split it into N sub-keys and sum on read. --- ## Real-world example: How Facebook's TAO graph cache works Facebook's TAO (The Associations and Objects) is a graph cache layer that sits between Facebook's web servers and MySQL databases. It serves the social graph — friends, likes, comments, photos — at a scale of billions of reads per second. **Key design decisions:** - **Objects and Associations:** TAO models data as objects (users, photos, posts) and associations (edges between objects — "user A likes photo B"). Associations are stored as directed edges with metadata. - **Cache-aside with write-through for associations:** When a user likes a photo, TAO writes to MySQL AND updates the cache in the same transaction. Reads never miss the cache for recently written associations. - **Per-region cache clusters:** Each Facebook data center has its own TAO cache cluster. Writes replicate asynchronously between regions. Within a region, reads from the local cache are sub-millisecond. - **Leader-follower cache topology:** Within a region, one cache shard is the leader (handles writes, replicates to followers). Followers handle reads. This gives strong consistency within a region and eventual consistency across regions. - **TTL hierarchy:** Objects (user profiles) cached for hours. Associations (likes, comments) cached for minutes. Counts (total likes) updated asynchronously — eventually consistent with TTL-based refresh. **The lesson:** Facebook doesn't cache everything forever. They use a tiered strategy: write-through for critical edges (ensuring a "like" is immediately visible to the user who made it), TTL-based invalidation for counts (stale counts are acceptable for seconds), and regional caching with async replication (users in different regions might see slightly different like counts for a few seconds — and that's fine). --- ## Common mistakes | Mistake | Why it's wrong | What to do instead | |---------|---------------|-------------------| | Caching without a TTL | Data lives in cache forever. Memory fills up. Stale data serves indefinitely. Every cached item needs an expiration | Always set a TTL. The TTL should be as long as the data's useful freshness window, with jitter to prevent avalanche | | Invalidating cache BEFORE the database write | Between invalidation and DB write, another request reads stale data from DB and populates the cache with it. Cache now has stale data indefinitely | Invalidate AFTER the database write commits. The tiny window between DB write and cache invalidation is safer than the reverse | | Using the same TTL for all keys | Keys expire simultaneously, causing cache avalanche. Some data needs 5-minute freshness, some needs 5-hour | Vary TTLs by data type. Add 10-20% random jitter. Profile access patterns to set appropriate TTLs | | Caching mutable data without an invalidation strategy | "I'll just set a short TTL" works until the TTL needs to be 1 second to meet freshness requirements — at which point the cache provides almost no benefit | If data must be fresh, use write-through or cache-aside with explicit invalidation on write. Short TTL is a band-aid, not a strategy | | Serializing entire objects to cache, then updating individual fields | A user object with 50 fields cached as JSON. You update one field (email). You must either: read-modify-write the entire blob (race condition) or invalidate and let the next read rebuild | Cache individual fields or use a cache that supports partial updates (Redis Hashes: `HSET user:123 email "new@email.com"`) | | Using Redis as a database, not a cache | No eviction policy set. Memory fills up. Redis starts rejecting writes. Application breaks because it assumed cache writes always succeed | Configure `maxmemory-policy allkeys-lru`. Redis should evict old keys when memory is full. It's a cache — data loss is acceptable by design | | Not monitoring cache hit rate | You added caching but never checked if it's working. 10% hit rate means 90% of requests pay the cache overhead (serialize, network, deserialize) for no benefit | Monitor hit rate per cache key pattern. A cache with <50% hit rate needs better key strategy or isn't worth the complexity | --- ## Production notes - **Use Redis Sentinel or Cluster for high availability.** A single Redis instance is a single point of failure. Sentinel provides automatic failover for simple Redis setups. Cluster provides sharding + failover. - **Monitor: hit rate, eviction rate, memory usage, latency percentiles.** A dropping hit rate means your caching strategy is degrading. A rising eviction rate means memory pressure — either add nodes or reduce TTLs/keys. - **Cache stampede protection is mandatory for hot keys.** If a key gets 10,000 QPS and goes cold, without stampede protection your database gets hit with 10,000 QPS instantly. Implement request coalescing (promise deduplication or Redis `SETNX`-based locking). - **Separate cache instances by workload.** Don't mix session storage (critical, can't lose), rate limiting counters (high throughput, ephemeral), and content cache (large values, TTL-based) on the same Redis instance. A memory-hungry content cache shouldn't evict user sessions. - **Pre-warm caches after deployment.** After a deploy, all application caches are cold. The first requests are slow. Prime caches by replaying production traffic (GoReplay) or running a warm-up script that preloads the top 1000 most-accessed keys. --- ## Performance notes - **Cache hit latency: Redis <1ms (same AZ), Memcached ~0.5ms, CDN 10-50ms, browser 0ms.** The more cache tiers a request hits before reaching the database, the faster the response. Measure cache latency at each tier. - **Compression trades CPU for bandwidth.** Gzip-compressing cache values can reduce size by 60-80%. For values >1KB, compression is usually worth it. Redis doesn't compress — compress in your application before `SET` and decompress after `GET`. - **Pipeline Redis commands for bulk operations.** Instead of 100 individual `GET` commands (100 round trips), use `MGET` (1 round trip) or pipelining (send 100 commands, read 100 responses in one batch). Latency drops from 100ms to ~2ms. - **Multi-level caching (L1 + L2) reduces Redis load by 40-80%.** An in-memory LRU cache in each application instance absorbs repeated reads of the same keys. Only cache misses reach Redis. The L1 cache size should be small (1000-10000 entries) to avoid memory pressure. - **Use Redis Hashes instead of JSON strings for partial updates.** `HSET user:123 email "new@email.com"` updates one field without reading the entire object. JSON requires read → parse → modify → serialize → write (race condition window + CPU overhead). --- ## Security notes - **Don't cache sensitive data without encryption.** Cached payment info, PII, or access tokens that live in Redis are readable by anyone with Redis access. Either don't cache sensitive data, or encrypt values with application-level encryption before caching. - **Redis authentication is not optional in production.** The default Redis config has no password. Set `requirepass` and use a strong, rotated password. Better: use ACLs (Redis 6+) to create limited users for different workloads. - **Never expose Redis to the public internet.** Redis has no transport encryption by default (TLS available in Redis 6+). An exposed Redis instance is a data breach waiting to happen. Security groups should restrict Redis to application subnets only. - **CDN cache poisoning is a real attack vector.** If your CDN caches responses based on unvalidated headers, an attacker can poison the cache with malicious content. Use `Vary` headers carefully. Validate `Host` headers. Cache only responses with explicit `Cache-Control` headers. - **Eviction policy matters for security.** If `maxmemory-policy` is `noeviction` and Redis fills up, writes fail. If your rate limiter depends on Redis writes and Redis is full, rate limiting stops — effectively allowing unlimited requests. Use `allkeys-lru` for cache workloads. --- ## Exercises ### Beginner 1. **TTL exercise:** Given a user profile that changes on average once per day, and a view count that changes 100× per second — what's an appropriate TTL for each? Justify based on freshness requirements and write frequency. 2. **Cache-aside implementation:** Write a TypeScript `cachedQuery` function that wraps a database query with Redis cache-aside. It should: check cache, return on hit, query DB on miss, populate cache with jittered TTL, and handle Redis connection failures gracefully (fall through to DB). ### Intermediate 1. **Implement cache stampede protection:** Write a `getOrSet` function using Redis that prevents cache stampede. Use `SETNX` to create a lock key when a cache miss occurs. Only the request that acquires the lock computes the value. Others wait and retry with exponential backoff (max 100ms). Handle lock timeout (don't let a crashed computation leave the lock forever). 2. **Build a two-tier cache:** Implement an L1 (in-memory LRU, max 5000 entries) + L2 (Redis) cache. On read: check L1 → check L2 → compute. On write: update L1 + L2 + invalidate across instances via Redis Pub/Sub. Measure the hit rate improvement over L2-only. ### Advanced 1. **Design a CDN caching strategy for a global e-commerce site:** The site has: product images (change rarely), product prices (change daily), inventory counts (change in real-time), user-specific recommendations (change per user). For each data type, specify: cache tier (browser, CDN, app, none), TTL, invalidation strategy, and how you handle the transition when a price changes (avoid showing old prices). 2. **Implement Facebook-style TAO cache for a social graph:** Design and implement a cache layer for a social graph with objects (users, posts) and associations (follows, likes, comments). Support: write-through for associations (a "like" is immediately visible to the creator), read-through for objects, association list queries (get followers of user X) with TTL-based freshness, and count queries (total likes on post Y) with eventual consistency. Use Redis with appropriate data structures (Hashes for objects, Sorted Sets for association lists by time). --- ## Interview questions ### Beginner 1. **Q: What are the three main cache failure modes at scale?** **A:** (1) **Cache Penetration:** requesting non-existent keys that bypass the cache and hammer the database. Fix: cache null/empty results (negative caching). (2) **Cache Avalanche:** many keys expire simultaneously, flooding the database. Fix: add random jitter to TTLs (TTL ± 20%). (3) **Cache Stampede (Thundering Herd):** a popular key expires and many concurrent requests try to recompute it simultaneously. Fix: request coalescing (only one request computes, others wait). 2. **Q: What's the difference between cache-aside and write-through caching?** **A:** Cache-aside (lazy loading) puts the application in control: check cache → if miss, load from DB and populate cache → on write, invalidate cache. The cache is populated on demand. Write-through: every write updates both the database and the cache synchronously. The cache is always consistent. Cache-aside is simpler and more common. Write-through is used when stale reads are unacceptable and write volume is moderate. 3. **Q: Why should you invalidate the cache AFTER the database write, not before?** **A:** Invalidating before the write creates a race condition: (1) Request A invalidates cache, (2) Request B reads from DB (still sees old data), (3) Request A writes to DB, (4) Request B populates cache with old data. Now the cache has stale data that won't be invalidated until the next write. Invalidating after the write closes this window — the worst case is a brief moment where the cache has new data and DB has old data (which resolves when the DB write completes). ### Senior 1. **Q: How would you design a caching layer for a system that needs to serve 1 million QPS with sub-10ms p99 latency?** **A:** This requires a multi-tier approach. **Tier 1 — CDN (10-50ms):** Cache static and semi-static content at the edge. For 1M QPS globally, CloudFront/Cloudflare absorb 60-80% of requests before they reach your infrastructure. **Tier 2 — L1 in-app cache (<1ms):** Each application instance maintains an LRU cache (e.g., 10,000 entries) for the hottest keys. This absorbs repeated reads of popular data without network hops. **Tier 3 — Redis Cluster (1-3ms):** Sharded Redis across multiple nodes using consistent hashing. With 10 Redis nodes handling 100k QPS each (well within Redis's capability on modern hardware), total throughput of 1M QPS is feasible. **Critical details:** use connection pooling, pipeline commands, and keep values small (<10KB) to avoid network bottlenecks. Use hash tags to co-locate related keys and enable `MGET`. Monitor per-shard hot spots and replicate hot keys. **The p99 target:** p99 <10ms means you need L1 hit rates >95% for the hottest keys, Redis in the same AZ (sub-2ms network), and aggressive timeout/fallback for Redis calls (fail open to DB after 5ms). 2. **Q: Explain how consistent hashing applies to a distributed cache cluster, and how virtual nodes prevent hot spots.** **A:** In a distributed cache, consistent hashing maps keys to cache nodes on a hash ring (0 to 2^32-1). `hash(key)` determines a position on the ring; the key is stored on the first node clockwise from that position. When a node is added or removed, only keys in its hash range move — typically 1/N of total keys. **Without virtual nodes:** each physical node has one position on the ring. When a node fails, its entire key range transfers to the next node clockwise. That single node now handles its own traffic + all of the failed node's traffic — a massive hot spot. **With 150 virtual nodes:** each physical node maps to 150 positions spread across the ring. When a node fails, its 150 virtual nodes distribute across ALL remaining nodes roughly evenly. Each remaining node gets ~1/N of the failed node's keys, not all of them. This prevents hot spots. Virtual nodes also help with heterogeneous hardware — give more virtual nodes to servers with more RAM. 3. **Q: Your production cache hit rate drops from 95% to 60%. Walk through your diagnostic process.** **A:** (1) **Check for cache evictions:** is `evicted_keys` spiking? If yes, memory pressure — TTLs too long, dataset growing, or a memory leak. Check `used_memory` trend. (2) **Check for key pattern changes:** did a deployment change cache key format? Are new keys being generated that don't match existing patterns? Check cache key cardinality. (3) **Check for cache stampede:** did a popular key expire and cause a cascade? Look for spikes in cache misses correlated with database load spikes. (4) **Check for network issues:** is latency to Redis increased? Timeouts causing fallback-to-DB? Check Redis slowlog and network metrics. (5) **Check for application changes:** did a new feature bypass the cache? Are writes now invalidating more keys? (6) **Segment the hit rate:** break down by key pattern (`user:*`, `product:*`, `session:*`). The drop might be isolated to one pattern. (7) **Check client-side:** are app instances restarting? Cold starts drop L1 hit rate to 0%. The fix depends on the root cause: increase memory, adjust TTLs, fix key patterns, add stampede protection, or scale Redis. --- ## Summary Caching is the most powerful performance tool in a backend engineer's arsenal — and the most dangerous when misunderstood. A multi-tier cache (browser → CDN → app memory → Redis → database) gives you orders-of-magnitude latency improvements, but each tier requires its own TTL and invalidation strategy. The three horsemen of cache failure — penetration (missing keys), avalanche (simultaneous expiry), and stampede (hot key expiry under load) — each have known countermeasures: negative caching, TTL jitter, and request coalescing. Redis Cluster with consistent hashing scales caching horizontally, but virtual nodes are essential to prevent hot spots. Facebook's TAO demonstrates the patterns at planetary scale: write-through for critical data, TTL-based for counts, and regional caching with async replication. Monitor your hit rate obsessively — a cache with <50% hit rate is slower than no cache at all, because you're paying the overhead without the benefit. --- ## Quick recall - Multi-tier cache: browser (<1ms) → CDN (10-50ms) → L1 in-app (<1ms) → L2 Redis (1-5ms) → DB. - Cache-aside: check cache, miss → load DB → populate cache. Write: invalidate cache AFTER DB write. - Write-through: write to DB AND cache synchronously. Cache always fresh. Cost: every write touches cache. - Cache penetration: fix with negative caching (cache nulls). Cache avalanche: fix with TTL jitter. Cache stampede: fix with request coalescing. - Redis Cluster: 16384 hash slots, `CRC16(key) % 16384`, consistent hashing for shards. Use hash tags for co-location. - Hot key problem: replicate key across shards, use L1 cache, or split into sub-keys. - Facebook TAO: write-through for associations, TTL for counts, per-region cache with async replication. - Always set TTLs. Never cache without eviction. Use `allkeys-lru` for cache workloads. - Monitor: hit rate, eviction rate, memory usage, latency p99. A dropping hit rate is your canary. - Never expose Redis to the internet. Use ACLs. Encrypt sensitive cached data. --- ## Quiz 1. **What is cache penetration, and how do you fix it?** A) Too many cache hits overwhelming the network B) Cache keys expiring simultaneously **C) Repeatedly requesting non-existent keys that bypass the cache — fix with negative caching (cache null results)** D) Redis running out of memory 2. **Which caching pattern ensures the cache is always consistent with the database?** A) Cache-aside **B) Write-through** C) Write-behind D) Lazy loading 3. **Why should you add jitter (random variation) to cache TTLs?** A) To make the system more unpredictable for attackers B) To comply with GDPR **C) To prevent cache avalanche — many keys expiring simultaneously and flooding the database** D) To reduce memory usage 4. **What does consistent hashing with virtual nodes prevent in a distributed cache?** **A) Hot spots — when a node fails, its keys distribute evenly across remaining nodes instead of overwhelming a single neighbor** B) Cache misses C) Network latency D) Data corruption 5. **What is the "thundering herd" (cache stampede) problem?** A) Too many users accessing the cache **B) A popular cache key expires and many concurrent requests all try to recompute it simultaneously, overwhelming the database** C) Redis Cluster failing over D) Too many cache keys being created 6. **Facebook's TAO uses write-through caching for what type of data?** A) User profiles **B) Associations (likes, comments) — ensuring immediate visibility for the user who performed the action** C) Friend counts D) News feed rankings 7. **Your Redis instance is configured with `maxmemory-policy noeviction` and reaches max memory. What happens?** A) Old keys are evicted automatically B) Keys with the nearest TTL are evicted **C) All writes are rejected — your application breaks if it assumed cache writes always succeed** D) Redis restarts automatically

Chapter 5 — Caching at Scale

Part of L5