Distributed Cache Architecture

The cache tier becomes its own distributed system

Once one cache node is not enough, the cache tier has to shard, replicate, and survive failures without becoming a new single point of failure or a new source of stale data. The starting decision is the engine.

Redis vs Memcached. Memcached is a lean, multithreaded, in-memory key-value store with LRU eviction and almost nothing else; it scales vertically across cores well and is ideal when you want a simple, fast, sharded blob cache. Redis is single-threaded per instance (for command execution) but gives you rich data structures, optional persistence (RDB snapshots, AOF log), replication, pub/sub, Lua scripting, and clustering. The crisp answer: pick Memcached when you want a pure, multi-core, evict-freely cache of opaque values; pick Redis when you need data structures, replication, persistence, or atomic operations (counters, rate limiters, leaderboards). Most systems reach for Redis and scale it horizontally by running many shards.

Sharding. Redis Cluster divides the keyspace into 16,384 hash slots; each key hashes (CRC16 mod 16384) to a slot, and slots are assigned to shards, so adding a shard means moving some slots rather than rehashing everything. The important property is consistent-hashing-style behavior: a topology change moves only a fraction of keys, avoiding a mass-miss event. Client-side sharding (a smart client hashing keys to nodes) is the Memcached equivalent.

Replication and HA. Each shard is a primary with one or more replicas. Replication is asynchronous, so a failover can lose the last few writes: acceptable for a cache, not for a system of record. Redis Sentinel (or Cluster's built-in failover) promotes a replica when a primary dies, so a node failure is a brief blip. The design principle that makes this safe: the cache is disposable. The source of truth is the database, so losing a cache node loses only performance, never data, as long as the application falls through to the DB on a miss.

Tiering. A remote cache is a network hop, too slow for the very hottest keys at high QPS. So you add an L1 near cache in the app process (a local LRU) in front of the L2 remote cache (Redis). L1 kills the hottest reads and shields Redis shards from hot keys. The cost of L1 is a second consistency layer: an invalidation now has to reach every app node's L1 (via pub/sub or a short L1 TTL), or you accept a small staleness window locally.

Consistency and operational hazards. Keep L2 in sync with the DB via invalidate-on-write, versioned keys (user:123:v7, so a stale value is simply never read), or a short TTL backstop. Under memory pressure, maxmemory plus an eviction policy (allkeys-lru) decides what leaves; a wrong policy (noeviction) turns a full cache into write errors. Two scale-specific hazards: a big key (a huge value or a million-element collection) blocks Redis's single thread when accessed or deleted and unbalances shards, so split it; and a hot key saturates one shard, handled with L1 and key replication.

Interview nuance: the flush trap. A cold cache is not safe to bring online under load, because every read misses and the full read volume hits the origin at once: the stampede across the whole keyspace. A cache restart, region failover, or FLUSHALL must be paired with cache warming or a gradual traffic ramp, with coalescing on. Treating a flush as free is the wrong turn interviewers listen for.

L1 near cache, sharded L2, and the disposable-cache principle

Step 1 / 3

Stage 1 of 3: The cache is disposable because the DB stays the source of truth: losing a cache node loses only performance, never data, as long as the app falls through to the DB on a miss.

Recap: pick Redis for structures/persistence/replication or Memcached for a lean multi-core blob cache, shard by hash slots so topology changes move few keys, replicate each shard with failover, tier L1-near plus L2-remote, keep L2 consistent via invalidate-on-write or versioned keys, and never bring a cold cache online under full load.