Caching Strategies: Multi-Level Cache Architecture and Advanced Redis

Caching keeps data in faster storage to cut response time and load on the origin. The design work is choosing a pattern and managing stampedes and consistency.

What Caching Solves

Caching keeps frequently accessed data in storage that is faster than the origin. It shortens response time and reduces load on the source of truth. A well-designed caching strategy can cut database queries and response times substantially, depending on the workload.

Redis is the de facto standard for in-memory caching, and combining multi-level caching with edge caching is a common approach. Preventing cache stampedes and maintaining cache consistency remain the hard parts of large-scale design.

Multi-Level Caching

Cache Layers

Characteristics of Each Layer

Problems Specific to Multi-Level Caches

Double stampede: If the in-process cache (L3) and Redis (L4) expire at the same moment, every instance goes through Redis to the database at once. That produces a stampede at two layers.

Fix: give L3 and L4 different TTLs, and keep the L4 TTL longer than the L3 TTL.

L3 TTL: 30 seconds (expires quickly, refreshed often)
L4 TTL: 300 seconds (safety net for L3 misses)

Caching Patterns

1. Cache-Aside (Lazy Loading)

The most widely used pattern. The application manages the cache and the database directly.

Read:
1. Look up the cache (GET key)
2. HIT -> return the cached data
3. MISS -> read from the DB -> store in the cache (SET key value EX ttl) -> return the data
 
Write:
1. Write to the DB
2. Invalidate the cache (DEL key)

async function getUser(userId: string): Promise<User> {
  // 1. cache lookup
  const cached = await redis.get(`user:${userId}`);
  if (cached) {
    return JSON.parse(cached);
  }
 
  // 2. DB lookup
  const user = await db.users.findById(userId);
  if (user) {
    // 3. store in the cache (TTL 5 minutes)
    await redis.set(`user:${userId}`, JSON.stringify(user), 'EX', 300);
  }
 
  return user;
}

Pros: caches only the data that is actually needed, and can fall back to the DB when the cache fails. Cons: the first request is always slow (cold start), and the cache can diverge from the DB.

2. Read-Through

The cache itself manages database access. The application only talks to the cache.

Read:
1. Request from the cache
2. HIT -> return
3. MISS -> the cache reads the DB itself -> stores -> returns

• Pros: simpler application logic
• Cons: the cache library or middleware has to contain the DB access logic

3. Write-Through

Data is written to the cache and the DB together.

Write:
1. Write to the cache
2. The cache writes to the DB (synchronously)
3. Respond once both have completed

Pros: the cache and the DB always stay consistent Cons: higher write latency (two writes), and data nobody reads still gets cached

4. Write-Back (Write-Behind)

Data is written to the cache immediately and to the DB asynchronously, later.

Write:
1. Write to the cache -> respond immediately
2. Write to the DB in the background (batched or delayed)

Pros: very fast writes, and DB load is spread out Cons: risk of data loss (unsynchronized data is gone if the cache fails)

Pattern Comparison

Cache Invalidation

"There are only two hard things in computer science: cache invalidation and naming things." - Phil Karlton

Invalidation is hard because deciding when and what to evict depends on the workload. The two families worth separating are TTL-based and event-based invalidation.

TTL (Time-To-Live) Based

Each cache entry gets an expiry time and is invalidated automatically.

SET user:123 '{"name":"Hong Gildong"}' EX 300  # expires automatically after 5 minutes

TTL guidelines:

• Frequently changing data: short TTL (30 seconds to 5 minutes)
• Rarely changing data: long TTL (1 hour to 24 hours)
• Data that must be invalidated on change: TTL plus event-based invalidation

Event-Based Invalidation

The cache entry is deleted explicitly when the data changes.

async function updateUser(userId: string, data: UpdateUserDto) {
  // 1. update the DB
  const updatedUser = await db.users.update(userId, data);
 
  // 2. invalidate the cache
  await redis.del(`user:${userId}`);
 
  // 3. invalidate related caches too
  await redis.del(`user-profile:${userId}`);
  await redis.del(`user-orders:${userId}`);
 
  return updatedUser;
}

Delete vs Update

Recommendation: in most cases the delete strategy is safer.

Cache Stampede Prevention

The Problem

When a popular key expires, hundreds or thousands of concurrent requests all hit the database directly and overload it. This is also called the thundering herd or dog-pile effect.

Cache expires -> 1,000 concurrent requests -> 1,000 identical DB queries -> DB overload

Prevention Strategies

1. Distributed Locking

Only one request queries the database; the rest wait until the lock is released.

async function getWithLock(key: string): Promise<any> {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);
 
  const lockKey = `lock:${key}`;
  // SET NX (set only if absent) - atomic lock
  const acquired = await redis.set(lockKey, '1', 'EX', 10, 'NX');
 
  if (acquired) {
    try {
      const data = await fetchFromDatabase(key);
      await redis.set(key, JSON.stringify(data), 'EX', 300);
      return data;
    } finally {
      await redis.del(lockKey);
    }
  } else {
    // wait for the lock and retry
    await sleep(50);
    return getWithLock(key);
  }
}

Collapsing concurrent recomputation into a single request, with a lock or a lease, cuts peak database load sharply. Facebook reported in 'Scaling Memcache at Facebook' (USENIX NSDI 2013) that peak DB query rate dropped from 17K/s to 1.3K/s, roughly 13x, after leases were introduced. The size of the reduction depends on key expiry patterns and concurrency.

2. Probabilistic Early Expiration

The entry is refreshed probabilistically before the TTL expires.

The XFetch algorithm attempts a refresh probabilistically ahead of expiry to prevent stampedes. delta is how long the recomputation takes, and beta tunes how aggressive the early refresh is. As the remaining TTL shrinks, the chance that it falls below the random value on the right-hand side grows, so only a fraction of requests refresh early just before expiry.

function shouldRefresh(ttlRemaining: number, delta: number, beta: number = 1.0): boolean {
  // XFetch algorithm
  const random = -delta * beta * Math.log(Math.random());
  return ttlRemaining < random;
}
 
// the smaller ttlRemaining is, the higher the refresh probability
// delta: time it takes to recompute the cache entry
// beta: tunes how early the expiry kicks in (default 1.0)

3. Singleflight / Request Coalescing

Concurrent requests for the same key are merged so that the database is queried once.

// Go's singleflight pattern
group := &singleflight.Group{}
 
result, err, shared := group.Do("user:123", func() (interface{}, error) {
    return fetchFromDatabase("user:123")
})
// shared == true means the result was shared with another caller

4. Comparison of Stampede Strategies

Advanced Redis

Redis Cluster

Data is distributed across 16,384 hash slots for horizontal scaling.

Slots 0-5460     -> Node A (+ replica A')
Slots 5461-10922 -> Node B (+ replica B')
Slots 10923-16383 -> Node C (+ replica C')

Redis Sentinel

A monitoring and automatic failover system for high availability (HA).

Failover flow: master failure detected -> Sentinel vote -> replica 1 promoted -> clients redirected

Advanced Redis Data Structures

Redis Example: Real-Time Leaderboard

# add or update a score
ZADD game:leaderboard 1500 "player_A"
ZADD game:leaderboard 2300 "player_B"
ZADD game:leaderboard 1800 "player_C"
 
# top 10 (descending by score)
ZREVRANGE game:leaderboard 0 9 WITHSCORES
 
# rank of a specific player
ZREVRANK game:leaderboard "player_A"
 
# query by score range
ZRANGEBYSCORE game:leaderboard 1000 2000

Cache Warming

The Idea

Cache warming fills the cache in advance so that load does not concentrate on the database during the cold start right after a service boots or a deployment lands.

Warming Methods

1. Startup warming: cache popular data while the service boots
2. Scheduled warming: refresh the cache periodically with a cron job
3. Event-based warming: cache immediately on a data change event (Change Data Capture, CDC)
4. Gradual cutover: ramp traffic to a new cache node slowly

The example below caches 1,000 popular products in one pass. redis.pipeline() batches the SET commands into a single round trip, and 'EX', 3600 sets a one-hour TTL on each key.

// startup warming example
async function warmCache() {
  const popularProducts = await db.products.findPopular({ limit: 1000 });
 
  const pipeline = redis.pipeline();
  for (const product of popularProducts) {
    pipeline.set(
      `product:${product.id}`,
      JSON.stringify(product),
      'EX', 3600
    );
  }
  await pipeline.exec();
 
  console.log(`Cache warming complete: ${popularProducts.length} products`);
}

Cache Consistency in Microservices

The Problem

In a microservice environment several services may cache the same data, and every one of those caches has to be synchronized when the data changes.

Approaches

1. Event-based invalidation: broadcast invalidation events over Kafka or Redis Pub/Sub
2. CDC-based automatic invalidation: detect DB changes with Debezium and refresh the cache
3. Short TTLs: settle for eventual consistency with short TTLs when strong consistency is not required
4. Cache tags: group related entries under a tag and invalidate them together

Edge Caching

Cloudflare Workers and Edge Caching

Serving the cache from the edge server physically closest to the user minimizes global latency. Cloudflare states on its network page that 95% of the world's internet users are within 50 ms of one of its data centers (Cloudflare Network, as of 2026). The closer the user is to an edge PoP, the faster a cache HIT comes back.

User (Korea) -> Seoul edge PoP -> cache HIT -> immediate response (< 50 ms)
User (Korea) -> Seoul edge PoP -> cache MISS -> origin server -> cache -> response

Edge cache strategy:

• Static content: images, JS/CSS -> long TTL (1 day to 1 year)
• Dynamic API responses: driven by Cache-Control headers -> short TTL (1 second to 5 minutes)
• Personalized content: not cacheable, or handled with the Vary header

Summary

Pick a caching pattern by trading off read and write performance against consistency and data loss risk. Start with cache-aside in most cases, move to write-through when strong consistency matters, and consider write-back when write throughput comes first. Guard against the stampede that follows the expiry of a popular key with distributed locks, probabilistic early expiration, or singleflight. In a multi-level cache, keep the L4 TTL longer than L3 to avoid a double stampede. Make delete the default invalidation strategy, and in microservices synchronize caches through events or CDC.

The next post in this series continues with another operations topic.

References

• Designing Data-Intensive Applications - Martin Kleppmann (Chapter 5)
• Redis Documentation: https://redis.io/docs
• System Design Interview - Alex Xu
• Cloudflare Workers Documentation: https://developers.cloudflare.com/workers