Empirical distributions and uniform bands

Let $X_1,\ldots,X_n$ be independent identically distributed real variables with cumulative distribution F. The empirical distribution $F_n(x)$ averages the indicators of observations at or below x. It is nondecreasing and right continuous, with limits zero and one at the two infinities. Repeated observations create larger jumps; they do not invalidate the definition. At a fixed nonrandom x, the count is binomial with success probability F(x), so the estimate is unbiased and has variance F(x)(1−F(x))/n.

A pointwise law of large numbers treats one threshold at a time. It does not by itself allow an arbitrary threshold selected after inspecting the entire curve. Uniform convergence instead controls the largest discrepancy anywhere on the real line. The Glivenko–Cantelli theorem gives almost-sure uniform convergence for this empirical distribution. Its applicability to arbitrary real distributions is a feature of the ordered threshold class; it is not a statement that every unrestricted family of measurable sets has a uniform law.

The one-sample Dvoretzky–Kiefer–Wolfowitz inequality with Massart's constant bounds the probability that $\sup_x|F_n(x)-F(x)|$ exceeds ε by $2e^{-2n\varepsilon^2}$. This finite-sample inequality holds for every real distribution F, including distributions with atoms. Continuity is needed for some exact Kolmogorov-statistic null distributions, but not for this upper bound. The result is stronger than applying one binomial interval and repeating its confidence label at every threshold.

Choose a failure probability alpha strictly between zero and one, solve the exponential bound for epsilon, and clip the resulting lower and upper curves to the probability range. With probability at least one minus alpha, every true cumulative probability is inside its interval simultaneously. The coverage probability is over repeated samples; once data are observed, it is not a posterior probability assigned to the fixed unknown distribution. The inequality can be conservative, especially at small sample sizes, and a radius exceeding one yields little information.

A simultaneous band remains valid at thresholds chosen from the data because its coverage event already includes all thresholds. It can also be inverted into conservative quantile statements using generalized inverses, with care at jumps and probability endpoints. For an interval probability, subtracting two cumulative bounds gives a valid bound after accounting for the correct endpoint convention. These transformations use monotonicity and the common uniform event, rather than assuming separate threshold errors are independent.

Neither the empirical curve nor its uniform band supplies a density band by differentiation. The empirical distribution is a step function, and differentiation is unstable in the uniform norm without additional regularity. Dependence, changing sampling distributions, selection bias, and censoring also require different arguments. A histogram of a correlated time series does not inherit this iid guarantee merely because its bins resemble an empirical distribution. Before calculating a confidence label, identify the sampling unit and the probability function being estimated.