Skip to main content

Replication Lag & Session Guarantees

Level 3: Level 3: Scaling the Data Tiermedium30 minsession-guaranteesreplication-lag

Map each lag-induced bug to its session guarantee and fix it with sticky routing or version tokens, instead of over-promising linearizability.

Lag is not a metric, it is a user-visible bug

Replication lag shows up as concrete, infuriating user bugs. You post a comment, the write hits the leader, your refresh reads a lagging replica that has not received it yet, and your comment vanishes. The fix is not "make replication synchronous everywhere" (too slow and defeats the point of replicas). The fix is to provide the specific, cheap session guarantee that the buggy interaction actually needs. There are four, and matching a bug to its guarantee is the skill:

  • Read-your-writes (read-after-write): after you write something, you always see it. Violated by the vanishing-comment bug. It only promises the writer sees their own write, not that others do.
  • Monotonic reads: you never see time go backwards. If read 1 shows a comment and read 2 (hitting a more-lagged replica) does not, the comment appears to un-happen. This is the "refresh and content disappears, refresh again and it comes back" flicker.
  • Monotonic writes: your own writes are applied in the order you issued them.
  • Writes-follow-reads (causal): if you read X and then write Y in reaction, everyone who sees Y also sees X (a reply never appears before the comment it replies to).
Check yourself

Each bug below violates one session guarantee. Match the symptom to the guarantee that would prevent it.

You post a comment, refresh, and your own comment is gone
A comment appears, vanishes on the next refresh, then comes back
A reply shows up before the comment it responds to
You update your bio, then still see the old one on the profile page

Two implementation techniques cover most cases

Sticky routing to the leader. For a bounded window after a user writes (say 10 to 30 seconds, or until the write is known to have propagated), route that user's reads to the leader or to a replica known to be caught up. Simplest read-your-writes fix. The catch is it is per-connection/per-session, so it breaks across devices: you write on your phone, read on your laptop with a different session, and the laptop still hits a lagging replica.

Version tokens (logical timestamps). On a write, the leader returns a version token (a log sequence number / LSN, a commit timestamp, or an opaque cursor). The client stores it and sends it with subsequent reads. The read path then waits for a replica to catch up to that token (or picks a replica already past it) before serving. This bounds staleness precisely and works across devices if the token travels with the user (in a cookie, the session store, or the client). It is how you get read-your-writes without pinning everything to the leader.

Interview nuance: be clear that these guarantees are strictly weaker than linearizability. Linearizability means a single, global, real-time order that every client agrees on; it is expensive (consensus, leader round trips, or reading from the leader with a read lease). Session guarantees only constrain what a single session or causal chain observes. The senior move is recognizing that the product almost never needs global linearizability; it needs "the user sees their own action," which read-your-writes delivers far more cheaply. Reserve linearizability for the few operations that truly need it (a uniqueness constraint, a distributed lock, a "claim this seat" check).

user writes comment -> LEADER (LSN=1042)  --returns token 1042-->
   later read carries token 1042 ->
      pick replica whose applied LSN >= 1042, else wait/route to leader
   => the write is never missing for this user (read-your-writes)

Recap: replication lag causes user-visible bugs, each violating a specific session guarantee; implement them with sticky routing to the leader (simple, single-device) or version tokens that make reads wait for a replica to catch up (works cross-device), and remember these are weaker than linearizability but usually exactly what the product needs.

Check yourself
Requirement: a user who posts from their phone must see that post when they open their laptop seconds later. What do you reach for?

Apply

Your turn

The task this lesson builds to.

Add read-your-writes and monotonic-reads guarantees to a read-replica architecture where a user writes to the primary and reads from lagging replicas.

Think about

  1. Which session guarantee does each user-visible bug violate?
  2. How do sticky routing and version tokens implement them?
  3. Why are these weaker than linearizability but often exactly enough?

Practice

Make it stick

A second problem on the same idea, so it survives past today.

Design the session-consistency layer for Twitter/X-style posting and timeline reads at 400k read QPS off async replicas, where a user posts from their phone and immediately opens the same account on their laptop, and neither device may ever show the tweet as missing.

Think about

  1. Why does sticky-to-primary routing fail the phone-then-laptop case?
  2. Where must the version token live so any device can honor it?
  3. What fraction of reads actually pay a wait, and why is that fraction small?