The requests you can see may differ from the requests that failed

Head sampling makes a retention decision early, while tail-oriented designs can use information accumulated later, such as latency or error status. Both can be useful, but their selection mechanisms answer different statistical questions. If errors are retained at a higher probability than successes, the fraction of retained traces that contain errors will overstate the population's error fraction when interpreted naively. That is an expected consequence of the design, not evidence that sampling failed to follow its intended rule.

Collection loss creates additional selection. A worker that crashes before exporting may leave no completed span; an exporter that drops under pressure may disproportionately lose the busiest instances. Record dropped-item counters, queue occupancy, sampling configuration, and instrumentation versions. An external request-count metric can help establish a denominator, but it too needs its own loss and semantics review. Combining two incomplete systems does not automatically recover the missing population if their gaps share a cause.

When inclusion probabilities are known and positive for all relevant units, inverse-probability weighting can estimate population totals under the sampling model. A selected item that had a low chance of inclusion represents more population weight. For a fixed known total request count, dividing an unbiased estimated error total by that fixed count preserves that total-based expectation property. Dividing by an estimated total is a ratio estimator and does not automatically have the same finite-sample unbiasedness.

Unknown or zero inclusion probabilities are a more serious limitation than a small sample. If all crashed requests have zero chance of being recorded and no independent evidence identifies them, weighting visible successes cannot reveal their outcomes. Likewise, a nominal sampler rate does not include undocumented exporter loss. State what the weighting corrects and what remains unidentified. Sensitivity ranges or explicit unknown counts can be more honest than a precise corrected percentage based on an incomplete selection model.

Report the visible numerator, denominator, weighting scheme, and known gaps alongside an impact estimate. Compare client-visible errors with server and dependency signals, especially when crashes prevent normal completion telemetry. For AI systems, also distinguish transport success from semantic failure: a fast fallback response with an empty or wrong result may not appear in an HTTP-error filter. The definition of error is part of the estimand and should not change silently during incident analysis.

The sampling lab shows what variability looks like under an intentionally unbiased iid model. Reset, set both p=0.1 and n=20, predict the exact sample-mean variance, then inspect and Play or Step. Reset and set both p=0.1,n=100 for a separate comparison. Variance falls from 0.0045 to 0.0009 under those assumptions. Playback highlights outcomes rather than collecting samples; the normal comparison and its CDF bound do not correct biased inclusion or certify individual bars. The real collection mechanism must be assessed separately.