Delivery Semantics: At-Most / At-Least / Exactly-Once

Three promises a pipeline can make

Level 5's distributed-systems theory already proved the theorem this lesson stands on: exactly-once delivery over a network is impossible, because a sender that never gets an ack cannot tell "message lost" from "ack lost" (the Two Generals problem). Take that as settled. The operational question here is the one the theory does not answer: given that impossibility, which of three promises does your pipeline actually make, and where does the ack sit that decides it?

Every message pipeline makes one of three promises, and stating which one, end to end, is the single most important sentence in an async design.

At-most-once: a message is delivered zero or one times. You never see a duplicate, but you can lose messages. You get this by acknowledging (committing your read position) before you process. If the consumer crashes after the commit but before the work finishes, that message is gone forever. Fine for a metrics sample or a best-effort log line, unacceptable for a payment.

At-least-once: a message is delivered one or more times. You never lose a message, but you can see duplicates. You get this by processing first and acknowledging after success. If the consumer crashes after processing but before the ack, the broker redelivers on restart and you process again. This is the practical default for anything that matters, because losing data is usually worse than repeating work, and repeats can be neutralized (see the idempotency lesson).

Exactly-once: every message takes effect once, no loss, no duplicate. This is what everyone wants and what the network cannot give you.

From impossible delivery to exactly-once processing

Since delivery itself cannot be exactly-once (the impossibility is stated above and proved in Level 5), the useful move is to redefine the goal. So what do the vendors mean by "exactly-once"? They mean exactly-once processing, achieved by taking at-least-once delivery and making the effect idempotent or transactional so that duplicates do not change the outcome. You convert a delivery guarantee into a processing guarantee at the consumer.

  producer --(at-least-once delivery, may duplicate)--> broker --> consumer
                                                                      |
                              [ idempotency / transaction here ]  <---+
                                                                      |
                                                            effectively-once effect

Interview nuance: what Kafka EOS actually covers

Kafka's "exactly-once semantics" (EOS) is real but narrowly scoped. It combines an idempotent producer (dedups producer retries into a partition using a producer id and sequence number) with transactions that atomically commit the consumer's read offset and the produced output records together. That gives exactly-once for a read-process-write loop that stays inside Kafka. It does not extend to external side effects. If your consumer sends an email, calls Stripe, or writes to a non-transactional database, Kafka EOS does nothing for those, and you must add an idempotency key yourself. Claiming Kafka gives you end-to-end exactly-once including third-party charges is the classic wrong turn.

Where the ack sits is the whole game: commit-before-process is at-most-once, process-before-commit is at-least-once. Pick at-least-once plus idempotency for anything with real-world consequences.

Recap: three guarantees (lose / duplicate / neither), exactly-once delivery over a network is impossible so you get exactly-once processing via idempotency or transactions, Kafka EOS is scoped to read-process-write inside Kafka only, and ack timing decides which guarantee you actually have.