Caching Strategy

Status: Complete
Category: Performance
Default enforcement: Advisory
Author: PushBackLog team

Tags

• Topic: performance, architecture
• Skillset: backend, frontend
• Technology: generic
• Stage: execution, review

Summary

Caching stores the results of expensive operations so they can be reused without repeating the work. A caching strategy defines what to cache, where to cache it (in-process, distributed, CDN, browser), how long to keep it, and when and how to invalidate it. Incorrect cache invalidation is one of the hardest problems in software engineering.

Rationale

The economics of caching

Database queries, external API calls, and compute-heavy transformations are expensive in time and cost. A result that takes 200ms to generate from the database takes microseconds to retrieve from a cache. For high-traffic systems, this difference translates directly into infrastructure spend and user experience.

Phil Karlton’s observation that “there are only two hard things in computer science: cache invalidation and naming things” is cited because it’s true. Caching introduces a consistency problem: the cached value may become stale when the source data changes. Every caching decision is a trade-off between freshness and speed.

Layers of cache

Caching exists at multiple layers in a modern system, and they compose:

• Browser cache: HTTP response caching; governed by Cache-Control headers
• CDN / edge cache: geographic distribution; terminates cache hits before they hit the origin server
• Application cache (in-process): in-memory store within a single server process (e.g., a Map or Caffeine)
• Distributed cache: shared across multiple server instances (Redis, Memcached)
• Database query cache: query result memoisation at the DB layer

When NOT to cache

Not everything should be cached. Poor caching choices create subtle, hard-to-diagnose bugs:

• User-specific or permission-sensitive data cached without user scoping can expose one user’s data to another
• Rapidly mutating data (real-time counters, live prices) with a long TTL causes stale reads
• Side-effecting operations (writes, transactions) must never be cached

Guidance

Cache placement decision

Cache invalidation strategies

HTTP Cache-Control

# Immutable static assets: can be cached forever (include hash in filename)
Cache-Control: public, max-age=31536000, immutable

# API response: cache 60s at CDN, validate with ETag after
Cache-Control: public, max-age=60, must-revalidate

# User-specific data: cache at browser only, not at CDN
Cache-Control: private, max-age=300

# Never cache
Cache-Control: no-store

Examples

Redis cache-aside with TTL

async function getProductDetails(productId: string): Promise<Product> {
  const cacheKey = `product:${productId}`;

  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);

  const product = await db.findProduct(productId);
  await redis.setex(cacheKey, 300, JSON.stringify(product)); // 5-minute TTL
  return product;
}

// Invalidate on update
async function updateProduct(productId: string, data: ProductUpdate): Promise<void> {
  await db.updateProduct(productId, data);
  await redis.del(`product:${productId}`); // Invalidate cached entry
}

In-process cache for reference data

const featureFlags = new Map<string, boolean>();
let flagsLoadedAt = 0;
const STALE_AFTER_MS = 30_000;

async function isEnabled(flagName: string): Promise<boolean> {
  if (Date.now() - flagsLoadedAt > STALE_AFTER_MS) {
    const flags = await loadFlagsFromDatabase();
    flags.forEach(f => featureFlags.set(f.name, f.enabled));
    flagsLoadedAt = Date.now();
  }
  return featureFlags.get(flagName) ?? false;
}

Anti-patterns

1. Cache without expiry

An entry cached forever becomes stale when the source data changes. Every cache entry should have a TTL or an explicit invalidation trigger.

2. Using cache to cover an N+1 problem

Caching the result of 50 individual queries is less efficient than fixing the query to use a join. Cache the result of the correct query, not each sub-query.

3. Caching security-sensitive data without scoping

A CDN cache that serves a user’s account page to all users because the cache key didn’t include the user ID. Always include the security-relevant scoping in the cache key.

4. Cache stampede / thundering herd

When a popular cache entry expires, thousands of concurrent requests all miss the cache simultaneously and hammer the database. Mitigate with probabilistic early recomputation, mutex locking, or stale-while-revalidate.

Related practices

• N+1 Query Prevention
• Lazy Loading
• Async Patterns

Part of the PushBackLog Best Practices Library. Suggest improvements →