advanced29 min studyLesson 399 in Engineering

An SLI starts with eligible events and a measurement boundary

Define good events, retries, deadlines, and coverage consistently so a reliability ratio reflects the promised user operation.

On this page

What you’ll understand

  • Specify an event-based SLI's eligible population, good predicate, and window.
  • Distinguish user operations from internal attempts and server observations.
  • Handle timeouts, absent traffic, and missing telemetry without turning unknown evidence into success.

A service-level indicator measures a defined aspect of service behavior, often as the fraction of eligible events that were good. The difficult part is usually deciding which events and whose observation count. A retry, a load-balancer rejection, a client timeout, or a missing measurement can change the population silently. Begin with the user promise, then make the instrumentation and query preserve its boundary instead of choosing whatever denominator is easiest to collect.

Define eligibility and goodness before writing a ratio

State the event unit: a user operation, an HTTP attempt, a completed task, or a requested byte range. Then define eligibility, including routes, tenants, environments, and any deliberate exclusions. Define good as a measurable predicate, such as a successful final result within a deadline. A status-only success indicator and a success-plus-latency indicator answer different questions. If expected invalid input is excluded, document the classification and verify it cannot hide genuine service failures. Eligibility changes should be reviewed as measurement-contract changes rather than quiet dashboard fixes.

The numerator must be a subset of the same denominator under the same window and source boundary. An internal server can miss failures before traffic reaches it, while a client can observe network or gateway behavior that the server never sees. A completion-only measurement can also miss work that never completes unless timeouts and abandoned operations are resolved according to policy. Choose where starts, terminal outcomes, rejections, and deadlines are recorded, then reconcile in-progress work. The measurement should reflect the promised experience rather than only the requests convenient for one process to count.

SLI⁡(W)=∑e∈EW1{G(e)}∣EW∣(∣EW∣>0)\operatorname{SLI}(W)=\frac{\sum_{e\in E_W}\mathbf{1}\{G(e)\}}{|E_W|}\qquad(|E_W|>0)
E_W is the defined eligible event population assigned to window W, and G is its good-event predicate. The ratio is dimensionless. Window assignment and terminal-outcome rules must be explicit; an empty denominator does not supply a mathematically defined success fraction.↔ Scroll wide equations horizontally.

Attempts, windows, and latency populations must agree

One user operation can generate several attempts, so an attempt-failure counter is not automatically a user-failure counter. Both are useful, but label the unit. Similarly, latency only for successful operations can look excellent while failed requests wait for long timeouts. Decide whether the indicator counts all eligible outcomes and which latency definition reflects the promise. Record duration using a suitable local timer at the chosen boundary, with consistent units and exactly one appropriate terminal classification per event. A retry log emitted several times must not multiply the denominator.

Align aggregation across instances and time. Pool good and eligible counts, not unweighted success percentages. Use compatible reset-aware rate or increase calculations for counter windows and retain the limits of sparse scrapes and lost accounting. Events that start before a window and finish after it require a defined assignment rule. Measurement coverage is separate from the indicator value: missing traffic, missing telemetry, and genuinely zero eligible traffic are different states. A default replacement of every missing series with zero can make an outage appear as an undefined or falsely healthy ratio.

Nbad=Neligible−Ngood,Ballowed=(1−S)NeligibleN_{\rm bad}=N_{\rm eligible}-N_{\rm good},\qquad B_{\rm allowed}=(1-S)N_{\rm eligible}
For a complete compatible event population and target fraction S between zero and one, the allowed bad-event budget is an event-count allowance. Observed bad count and budget use the same event unit; this arithmetic does not fix unknown eligibility, missing data, or a measurement-point mismatch.↔ Scroll wide equations horizontally.

Validate the instrumentation with adversarial histories

Use known fixtures for a direct success, a retry that eventually succeeds, a final failure, an upstream rejection, an exceeded deadline, cancellation, and a request whose telemetry export fails. Predict each fixture's eligible and good contributions before checking the dashboard. Compare client and server measures to explain boundary differences rather than forcing them to agree. Include a no-traffic interval and a collection outage as distinct tests. Keep SLI metric collection separate from outcome-enriched trace selection unless a documented valid estimator and denominator supply the promised population measurement.

The sampling lab illustrates a hypothetical good-event fraction with strong independence assumptions. Reset, set both p=0.97 and n=100, predict mean 0.97 and variance 0.000291, then Play or Step. The exact binomial distribution is not an observed production window and playback does not draw random events. Its normal overlay is approximate, and its Berry–Esseen statement is about standardized CDF error rather than each bar. Real traffic can be correlated and systematically unmeasured, so the instrument cannot certify SLI coverage or turn a chosen p into evidence of actual reliability.

A retry changes the attempt denominator

A complete client-boundary window has 1,000 eligible user operations. Ultimately 990 succeed, and 970 both succeed and finish within 300 ms. Internal retries produce 1,200 attempts: 990 successful and 210 failed attempts.

  1. The outcome-only user success fraction is 990/1,000 = 99 percent, using final user outcomes rather than attempts.

  2. The success-within-deadline SLI is 970/1,000 = 97 percent. Its thirty bad user operations include final failures and otherwise successful operations that missed the deadline.

  3. The internal attempt success fraction is 990/1,200 = 82.5 percent. It is useful retry-pressure evidence but is a different population and cannot replace either user-operation SLI.

The result

The same system has several valid ratios with different questions. Naming event units and good predicates prevents an attempt metric from silently redefining the user promise.

Explore the model

Interactive lab

Interactive lab · sampling

An assumed good-event model is not measured coverage

Reset, set both p=0.97 and n=100, predict good-fraction mean 0.97 and variance 0.000291, then Play or Step. Exact binomial masses assume independent common-probability events and are not random production observations. The normal overlay is approximate; its Berry–Esseen bound concerns standardized CDFs. No eligibility, timeout, or collection-coverage logic is modeled.

An assumed good-event model is not measured coverageExact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Current readings: Mean sample proportion: 0.5; Sample-mean variance: 0.0125; Uniform CDF error bound: ≤ 0.1062. All parameters are adjustable with the labeled controls below.Exact distribution of the mean of 20 independent Bernoulli variables with success probability 0.5; playback highlights outcome 0.00.250.50.75100.0580.1150.1730.23Sample mean · k / nProbability massNormal bin masses are an approximationk = 0 · exact probability = 9.54e-7
Exact binomial massNormal interval massHighlighted outcome / mean
Mean sample proportion
0.5
Sample-mean variance
0.0125
Uniform CDF error bound
≤ 0.1062
t = 0 s

Exact binomial masses for the mean of n independent Bernoulli trials, compared with normal mass between adjacent half-count boundaries; normal tails outside the finite support remain outside the bars. Playback highlights outcomes, not random samples. The Berry–Esseen bound applies to standardized CDFs, not individual bars or continuity-corrected errors. At p = 0 or 1, normal comparison and that bound are disabled. Normal-CDF evaluation uses a numerical approximation with absolute error below 7.5×10⁻⁸ (at most 1.5×10⁻⁷ per interval, before rounding). Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forAn SLI needs measured eligible events and a correct boundary; a probability chosen in a teaching model cannot establish reliability.

Check your understanding

An operation retries twice before a successful final response, and the SLI is defined over final user operations. How should it affect that SLI's eligible-event denominator?

Try it yourself

A complete window has 20,000 eligible operations and 19,920 good ones. Compute the SLI and observed bad count. For a target of 99.9 percent, compute the allowed event budget and compare counts.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Use one minus the target fraction for the budget, and preserve the same eligible-event denominator.

Compare with the solution

The SLI is 19,920/20,000 = 99.6 percent and bad count is eighty. A 99.9 percent target permits 0.001 × 20,000 = twenty bad events for this complete population. Eighty is four times that allowance. Missing or inconsistent measurement would require separate treatment rather than being fixed by this arithmetic.

Take it with you

  • Define the event unit, eligibility, good predicate, window, and measurement boundary together.
  • Retries and latency-only-success views can silently change the population being reported.
  • Validate timeouts, no traffic, and telemetry loss separately before treating a ratio as a reliability measure.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.