Sagas: Orchestration vs Choreography & Compensation

Local transactions, compensating undos

When a business transaction spans services and 2PC is too blocking, the standard answer is a saga: a sequence of local transactions, one per service, where each step has a compensating action that semantically undoes it. There is no global lock and no global commit. You make forward progress step by step, and if a later step fails you run the compensations for the steps that already succeeded, in reverse. A saga gives you atomicity of outcome (fully done or fully undone) but crucially not isolation.

Orchestration vs choreography

• Orchestration: a central orchestrator (a Temporal/Cadence workflow, an AWS Step Functions state machine) explicitly calls step 1, step 2, step 3, and on failure invokes the compensations. Pros: the flow lives in one place, easy to reason about, trace, and add timeouts/retries. Cons: the orchestrator is a component you must run and make reliable.
• Choreography: no central brain. Each service listens for events and reacts: Order emits OrderCreated, Inventory reacts and emits InventoryReserved, Payment reacts, and so on. Pros: highly decoupled. Cons: the end-to-end flow is implicit, scattered across services, hard to trace and debug, especially for compensations. Cyclic event dependencies sneak in.

Rule of thumb: choose orchestration for anything with more than a couple of steps, non-trivial compensation logic, or where on-call must be able to see "where is this order stuck?" Choose choreography only for short, simple, truly decoupled flows.

The interview-critical property: no isolation

Between steps, intermediate states are visible to other transactions. In an order saga, inventory is reserved (visible) before payment succeeds; another request can observe "reserved but unpaid." This is a real anomaly a single ACID transaction would never expose. Manage it with countermeasures:

• Semantic lock: mark a record with a pending/in-saga flag (order status PENDING) so others treat it as tentative.
• Commutative updates: design operations so order does not matter (increment/decrement rather than absolute set).
• Reread / version check: verify a version/state before compensating, so you compensate against current reality, not a stale snapshot.

Compensations are their own hazard

A compensation must be idempotent (it may be retried) and it may itself fail. "Un-charge a card" is fine as a refund, but "un-send an email" or "un-ship a package" is not truly reversible, so you compensate semantically (issue a recall, send an apology, restock on return). For compensations that fail: retries with backoff, a dead-letter queue, and ultimately operator escalation. This durability and retry machinery is exactly what Temporal / Step Functions give you for free.

Interview nuance: the two things interviewers probe are (1) "sagas give atomicity but not isolation, what anomaly does that allow and how do you contain it?" and (2) "what happens when a compensation fails?" Concrete answers to both put you ahead of most candidates.

Recap: a saga chains local transactions each with a compensating undo, coordinated centrally (orchestration, preferred for anything non-trivial) or via events (choreography); it guarantees the outcome is all-or-nothing but exposes intermediate state, so you add semantic locks and version checks, and make compensations idempotent with retries, DLQ, and escalation.