Predictive versus parameter uncertainty

Condition on a dataset D and an input x. In a Bayesian description, a parameter Θ is distributed according to a posterior, and the future outcome Y is distributed conditionally on Θ. Integrating over Θ produces a predictive distribution. An interval for a parameter and an interval for a future observation answer different questions, even when both use posterior samples. For scalar regression, total variance separates expected conditional outcome variance from variance of conditional means, provided the second moments exist.

The first component is often called aleatoric uncertainty and the second epistemic uncertainty. Those labels require context. A sensor effect treated as noise in one model could become predictable after adding a measured covariate in another. Conversely, a posterior can concentrate tightly around the wrong model. A small parameter component then reflects agreement within a restricted family, while systematic prediction errors remain large. Record the variables observed, likelihood, prior, fitting procedure, and units before interpreting either component as reducible or irreducible.

For binary outcomes, suppose each model assigns probability pΘ to label one. The marginal predictive probability is the posterior mean of pΘ. Its Bernoulli variance can be decomposed into expected within-model Bernoulli variance and variance across those probabilities. Models that all say one half and models split between zero and one have identical single-outcome marginal predictions, yet opposite allocations between the two terms. The decomposition therefore needs access to the chosen distribution over models, not just the averaged output.

Deep ensembles and repeated dropout passes can expose useful disagreement, but disagreement is not automatically an exact posterior variance or a reliable detector of every unfamiliar input. Shared architecture, training data, and optimization choices can make all members confidently wrong together. Approximate Bayesian interpretations also have assumptions about priors, objectives, and variational families. Evaluate whether a proposed uncertainty score predicts the errors relevant to the application, and compare it against simple baselines rather than promoting the method name into a guarantee.

When disagreement is high, collecting labels near the disputed inputs may help distinguish candidate models. When every model predicts a broad outcome distribution, better measurements or new features may matter more than adding nearly identical examples. Test these hypotheses by tracking held-out prediction quality after targeted interventions. Preserve the before-and-after data definitions: a narrower reported interval caused by a different posterior approximation is not necessarily an improvement in accuracy, coverage, or useful decision quality.

The probability instrument exposes only a marginal law. Reset, set p to 0.5 and n to 10, predict mean five and variance 2.5, then Step. Compare a fresh Reset with both p at 0.5 and n at 20 before Step: mean ten, variance five. These are exact independent-binomial counts. The lab has no posterior, model ensemble, or shared latent parameter and cannot identify either uncertainty component from the displayed probability.