SLO and error budget

David Granzin By David Granzin

A service level objective (SLO) is a target for how often a service must behave well over a time window, such as 99.9% of requests succeeding over 30 days. The error budget is the failure that target allows: 0.1% of requests.

SLI, SLO and SLA

TermWhat it isExample
SLI, service level indicatorThe measurement: a ratio of good events to all eventsRequests answered without a 5xx in under 500 ms, divided by all requests
SLO, service level objectiveThe target for that ratio over a window99.9% over 30 days
SLA, service level agreementA contract with consequences, usually looser than the SLO99.5% per month, or service credits

The SLI is defined from the user's side. "CPU under 80%" is not an SLI, because users do not experience CPU. "Checkout requests that succeed" is one.

The error budget, with numbers

The error budget is 1 − SLO. A 99.9% target leaves 0.1% for failure. Over 30 days, measured in time, that is 43.2 minutes of full downtime. Measured in requests, which is usually more accurate, a service handling 10 million requests a month may fail 10,000 of them.

SLOBudgetPer 30 days
99%1%7 h 12 min
99.5%0.5%3 h 36 min
99.9%0.1%43 min 12 s
99.95%0.05%21 min 36 s
99.99%0.01%4 min 19 s

The budget turns reliability into a decision. While budget remains, the team ships features and takes risks. When it runs out, reliability work comes first until the window recovers. Neither side has to argue from opinion.

Burn rate and alerting

Burn rate is how fast you are spending the budget compared with spending it evenly. A burn rate of 1 uses the whole budget exactly at the end of the window. A burn rate of 14.4 for one hour spends 2% of a 30-day budget, which is worth waking someone for.

Alerting on burn rate instead of raw error rate cuts noise. A one-minute spike that costs 0.01% of the budget does not page anyone; a slow, steady 3x burn that would exhaust the budget in ten days does. The common setup pairs a fast window (such as 1 hour at 14.4x) with a slow one (such as 6 hours at 6x).

Where SLI data comes from

Request-based SLIs come from server-side metrics or spans: count requests and count the good ones. Two cautions. Counting from sampled traces undercounts unless each span carries its sampling weight, as the sampling guide explains. And a latency SLI built on a percentile is harder to turn into a budget than one that counts requests over a threshold.

Sources

See your first trace today.

Add the SDK, point your OTLP exporter at Maple, and traces arrive.

maple.dev: observability on OpenTelemetry