Design a Payment System & Ledger

"Roughly correct" is a failing answer

Payments is the interview where "roughly correct" is a failing answer. The whole problem is money that must never be double-charged, never lost, and always auditable, and every design choice flows from that. Volume is modest by web standards (a large processor might do 5K to 50K payments/sec at peak), so this is a correctness problem, not a throughput problem.

Idempotency, because retries are guaranteed

Networks time out, clients resubmit, and your own workers retry after crashes. Every mutating request carries a client-generated idempotency key (a UUID the client mints per logical intent). The payment service stores that key with the request result in a dedup table before doing work, keyed uniquely so a second request with the same key returns the first result instead of charging again. This turns at-least-once delivery into effectively-once behavior. Without it, one dropped ACK becomes a double charge.

Double-entry, immutable ledger

The ledger is the source of truth, and it must be double-entry and immutable. Instead of storing a mutable balance column you update in place, you append immutable journal entries: every movement of money is two entries that sum to zero (debit one account, credit another). A charge of $50 becomes a debit to the customer's funding account and a credit to the merchant's payable account. A balance is then a derived sum of entries, never an overwritten field. This gives you a complete audit trail, makes reconciliation with the bank statement mechanical, and makes bugs detectable (entries that do not sum to zero are corruption you can alarm on).

Table

The 50 dollar charge as two immutable journal entries that sum to zero. A balance is a derived SUM over an account's entries, never an overwritten column.

entry	account	side	amount
jrnl-1	customer funding account	debit	-50.00
jrnl-2	merchant payable account	credit	+50.00
''	sum of entries	''	0.00

Interview nuance: the fastest way to fail this round is proposing UPDATE accounts SET balance = balance - 50. Say explicitly that you use an append-only double-entry ledger and derive balances, because mutable balances make audit and reconciliation impossible and hide bugs.

Coordinating across systems with a saga

A charge spans several systems (your wallet/ledger, an external provider like Stripe or Adyen, and the orders service), and you cannot hold a distributed ACID transaction across an external API. Use a saga (an orchestrated sequence of local transactions with compensating actions). The orchestrator: (1) reserves funds in the ledger as a pending entry, (2) calls the provider with an idempotency key, (3) on success posts the settled ledger entries and marks the order paid, (4) on failure posts a compensating reversal. State lives in a durable workflow so a crash resumes rather than orphans money.

client --idem key--> Payment API --> dedup check
   |                                     |
   v                                     v
saga orchestrator --> ledger (pending) --> provider (charge, idem) --> ledger (settle) --> order paid
        \--- on failure ---> compensating reversal entry ---/

Providers confirm asynchronously via webhooks, which are themselves at-least-once, so webhook handlers must be idempotent too (dedup on the provider's event id). Reconcile daily by summing ledger entries and comparing to the provider's settlement report; any drift is an incident. Layer PCI scope reduction (never store raw PANs, tokenize via the provider) and fraud hooks on top.

Recap: idempotency keys on every mutating call turn retries safe, an append-only double-entry ledger with derived balances gives auditability and reconciliation, and a saga with compensations plus idempotent webhook handling coordinates the provider, wallet, and orders without a distributed transaction.