Redis Caching Strategies Every Backend Engineer Should Know

TL;DR Redis is a high-performance, in-memory data store that is widely used as a cache layer. To get the most value from Redis you must choose appropriate caching strategies (cache-aside, read-through, write-through/behind, negative caching, TTL policies, eviction strategies), handle concurrency and cache stampedes, design for memory efficiency, ensure HA and correct shard/cluster design, and use Redis features (Lua scripts, modules, pub/sub, Streams) to implement robust, consistent behavior. This article provides a deep-dive into these strategies, trade-offs, code examples, and operational best practices for production systems.

Table of contents

• Background and brief history
• Core Redis concepts relevant to caching
• Why caching matters (metrics & goals)
• Caching strategies (detailed)
• Cache-aside
• Read-through
• Write-through
• Write-behind (write-back)
• Write-around
• TTL & eviction strategies
• Negative caching
• Cache stampede mitigation
• Cache invalidation strategies (lazy, eager, tagging)
• Redis-specific features & patterns to implement strategies
• Atomic operations & Lua scripts
• Hashes for memory-efficiency
• Modules: RedisJSON, RedisBloom, RedisGears, RediSearch
• Pub/Sub & Streams for cache invalidation
• Sharding, clustering, and high availability
• Data modeling & serialization for efficient caching
• Monitoring, testing, and benchmarking
• Security, deployment, and operational best practices
• Practical examples / code snippets
• Case studies and real-world patterns
• Future directions & trends
• Best-practice checklist
• Further reading

Background and brief history

Redis (REmote DIctionary Server) was created by Salvatore Sanfilippo in 2009. It evolved from a simple key-value store into a rich in-memory data platform with multiple data types (strings, hashes, lists, sets, sorted sets), powerful commands, persistence options (RDB, AOF), replication, cluster mode, Sentinel for HA, and an extensible module system. Redis’ sub-millisecond performance and flexible data model make it ideal as a caching layer for backend systems.

Core Redis concepts relevant to caching

• Data types: strings, hashes, lists, sets, sorted sets (ZSET), bitmaps, HyperLogLog, streams.
• TTL and expiration: EXPIRE, PEXPIRE, EXPIREAT.
• Eviction policies (when maxmemory exceeded): noeviction, allkeys-lru, volatile-lru, allkeys-lfu, volatile-lfu, volatile-ttl, volatile-random, allkeys-random.
• Persistence: RDB (snapshotting), AOF (append-only file) — trade-offs for durability vs speed.
• Replication and HA: master-replica replication, Sentinel for failover, Cluster for sharding and scaling.
• Atomicity: single commands are atomic; Lua scripting provides multi-command atomicity.
• Pub/Sub, Streams, and modules (RedisJSON, RedisBloom, RedisTimeSeries, RedisGraph).

Why caching matters (metrics & goals)

Primary goals:

• Reduce latency for end users.
• Reduce load on primary storage (databases, external APIs).
• Improve throughput and reliability.

Key metrics:

• Cache hit rate = hits / (hits + misses)
• Latency (p50/p95/p99)
• Throughput (ops/sec)
• Eviction rates
• Memory usage and fragmentation
• Errors (timeouts, timeouts to origin)

Trade-offs: consistency vs performance, memory cost vs hit rate, complexity vs predictability.

Caching strategies (detailed)

1) Cache-aside (Lazy loading)

Pattern:

• On read: application checks cache. If miss, fetch data from DB, populate cache, return result.
• On write: update DB, then invalidate/delete or update cache.

Pros:

• Simple, flexible, minimal coupling to Redis.
• Cache only what is needed.

Cons:

• Potential for stale reads unless write path invalidates cache correctly.
• Cache stampede on high concurrency for popular keys.

When to use:

• General-purpose caching for queries or objects where eventual consistency is acceptable.

Example (pseudo):

• GET key from Redis
• If found, return
• Else fetch from DB, SET key with TTL, return

2) Read-through

Pattern:

• Cache is augmented with logic (e.g., a caching proxy or a library) so that reads go to cache and cache handles fetching from DB on miss.
• Application reads from cache only.

Pros:

• Centralized responsibility for cache filling.
• Can simplify application code.

Cons:

• Adds coupling, may increase latency if the caching layer synchronously fetches DB.

When to use:

• When using client libraries or infrastructures that support read-through (e.g., some caching frameworks).

3) Write-through

Pattern:

• All writes go via the cache. The cache writes synchronously to the backing store before acknowledging.
• Ensures cache and DB are synchronized at write time.

Pros:

• Stronger consistency: cache and DB are always in sync at write completion.

Cons:

• Higher write latency (need to write DB in write flow).
• More load on DB (writes can’t be batched easily).

When to use:

• When consistency on writes must be maintained, and write latency is acceptable.

4) Write-behind (Write-back)

Pattern:

• Writes are made to the cache and acknowledged immediately; the cache asynchronously persists to the DB later.
• Can batch multiple writes to reduce DB load.

Pros:

• Lower write latency, can batch writes for efficiency.

Cons:

• Risk of data loss if cache fails before persisting changes.
• More complex: handling order, retries, batching semantics.

When to use:

• When write latency is critical and occasional loss is acceptable or when cache durability mechanisms are strong.

5) Write-around

Pattern:

• Write operations bypass cache and go directly to DB. Read populates cache on misses.
• Helps avoid caching items that are written once and never read.

Pros:

• Keeps cache space focused on popular reads.

Cons:

• If written items are read soon after, read path will miss and fetch from DB.

6) TTL & Expiration

Using TTL reduces staleness and bounds memory usage. Common approaches:

• Fixed TTL: e.g., 5m for all cache keys.
• Sliding TTL (touch on access) for LRU-like behavior.
• Adaptive TTLs per key type or user.
• Stochastic TTLs (add jitter to TTLs) to avoid mass expiration (thundering expiration).

Commands: EXPIRE, PEXPIRE, SET key value EX px NX, etc.

7) Eviction policies

When Redis reaches maxmemory, it evicts keys according to configured policy. Choose based on access pattern:

• allkeys-lru: evict least recently used among all keys.
• volatile-lru: LRU among keys with TTL only.
• allkeys-lfu: evict least frequently used—good when popularity distribution is stable.
• volatile-ttl: evict keys with nearest expiration.
• noeviction: reject writes when out of memory.

Considerations:

• LRU approximations in Redis are not perfectly precise but good enough.
• You must set maxmemory and an eviction policy that fits your data characteristics.

8) Negative caching

Cache "not found" or "null" responses with a short TTL to avoid repeatedly hitting DB for missing items. Use short TTL to allow eventual creation.

Example:

• If DB returns no row, cache a special tombstone value (e.g., "NULL") for 60 seconds.

9) Cache stampede (thundering herd) mitigation

Problem: many clients attempt to fill cache simultaneously on expiration leading to high origin load.

Solutions:

• Locking / singleflight: coordinate so only one client fetches, others wait for result (SETNX-based lock or distributed lock).
• Request coalescing: queue waiters until the result is available.
• Probabilistic early expiration: expire keys slightly before TTL with random offset so requests are spread out.
• Stale-while-revalidate: return stale data while a background refresh happens (dual TTL: fresh TTL + stale TTL).
• Leases: a client owns the right to refresh for a period (GCache's approach).
• Use Redis’ built-in features: Lua script to perform get-or-set atomically.

10) Cache invalidation strategies

• Time-based: rely on TTLs (cheap, eventual).
• Explicit invalidation: application invalidates cache on writes (DELETE or SET overwrite).
• Versioning: store version token; on change bump token so stale keys are ignored.
• Tagging (group invalidation): maintain sets of keys per tag to invalidate group of items.
• Event-driven: use pub/sub or Streams to publish invalidation events that subscribers apply.

Redis-specific features & patterns to implement strategies

Atomic get-or-set using Lua

Lua script ensures that a get/miss/fetch/set sequence is atomic to avoid races.

Example Lua script (simple): ``lua -- KEYS[1] = key, ARGV[1] = ttl (seconds), ARGV[2] = value-if-miss (string) local val = redis.call("GET", KEYS[1]) if val then return val end redis.call("SET", KEYS[1], ARGV[2], "EX", ARGV[1]) return ARGV[2] ``

In practice, you’d have the application fetch the value from origin if Lua doesn't contain it; but you can use Lua to set a placeholder lock.

Distributed locking (SET NX EX)

Use SET key value NX PX milliseconds to implement locks. Example: ``text SET lock: NX PX 3000 `` On refresh or release check that token matches to avoid releasing another client's lock.

Note: Redlock (algorithm by Redis author) has controversy. Use simple locks or libraries (e.g., Redisson for Java) if you understand the trade-offs. For single Redis instance, SET NX is typically enough. For correctness across multiple nodes, careful consideration is required.

Singleflight / request coalescing

Libraries exist in several languages:

• Go: golang.org/x/sync/singleflight
• Node/JS: p-memoize or custom constructions
• Use Redis to signal refresh in progress and a list of waiting clients, ...