Error Budgets & Policy

The error budget is permission to fail

An error budget is the inverse of your SLO: budget = 1 - SLO. A 99.9% SLO means 0.1% of requests are allowed to fail, and that 0.1% is a real, spendable resource. Over 28 days at 10 million requests, a 99.9% SLO buys you 10,000 failed requests, or about 43 minutes of full outage. The mental shift that makes this powerful: the budget is not a threat, it is permission to fail that much. It exists to be spent, not hoarded.

Why spent, not hoarded? A team sitting at 100% budget remaining all quarter is not "doing great," it is over-investing in reliability the users did not ask for and under-shipping features they did. Perfect reliability is the wrong target because it means you shipped too slowly. The budget converts reliability from an argument ("is this safe enough?") into an account balance everyone can read.

The policy is the point

A pre-agreed, written set of consequences that trigger automatically as the budget drains, so the ship-versus-stabilize decision is made in advance and does not become a political fight during a crisis. A typical policy:

Budget remaining   Consequence
100% - 50%         Normal operation. Ship features at full speed.
50% - 10%          Caution. Extra review on risky changes; start
                   burning down reliability debt in parallel.
0% (exhausted)     Feature freeze. All release capacity redirects to
                   reliability until the budget recovers.

The freeze is the teeth. When the budget hits zero, feature launches stop and the team works reliability until the rolling window recovers the budget. This is what makes the SLO enforceable rather than aspirational.

Two carve-outs keep the policy sane. First, security and P0 fixes ship even during a freeze: a freeze must never block a patch for an actively exploited CVE or a data-loss bug. Second, the freeze applies to new features and risky changes, not to reliability work itself. The point is to redirect effort, not to halt all deploys.

Interview nuance: the policy only works if it depoliticizes the decision and has shared accountability. Dev and ops (or product and SRE) both sign the policy in advance, and leadership pre-commits to honoring the freeze. Without that pre-agreement, when the budget is blown the product VP will simply overrule the freeze for the quarter's big launch, and the SLO becomes theater. The budget's whole purpose is that nobody has to win that argument in the moment: the number already decided.

Track burn over a rolling window and remember that one bad incident can consume weeks of budget. A 90-minute outage against a 43-minute monthly budget doesn't just fail the month, it can put you underwater for the next two windows. That is why the response to a blown budget is a freeze, not a shrug: you are already borrowing against the future.

Recap: error budget = 1 - SLO and it is permission to fail that you spend, not hoard; the policy pre-agrees consequences (freeze at zero) with security carve-outs and shared accountability so the ship-versus-stabilize call is depoliticized before the incident, not fought during it.