core23 min studyLesson 19 in Mathematics

Statistical inference

Separate a sample from a model, quantify estimation uncertainty, and distinguish confidence from posterior probability.

Before you begin

Random variables and distributions

On this page

The question for today

What you’ll understand

Distinguish an unknown parameter from a random estimator.
Interpret a confidence interval using repeated sampling.
Explain how a Bayesian update combines likelihood and prior assumptions.

Probability predicts data from a model; inference uses data to learn uncertain model parameters. Many parameters can produce the same finite sample. A point estimate compresses that ambiguity, while an interval or posterior describes it. Good inference states its assumptions and the meaning of uncertainty. Decimal precision cannot repair an unrepresentative sample.

Parameters describe a model; estimators use data

A parameter is fixed but unknown within a frequentist model, such as a Bernoulli success probability. An estimator converts a random sample into a guess. Before sampling, the estimator is random; afterward, its observed estimate is a number. This separation lets us judge the rule across repeated samples instead of trusting one convenient outcome.

Likelihood evaluates how compatible different parameter choices are with the observed data, holding that data fixed. For independent Bernoulli observations, the maximum-likelihood estimate of the success probability is the observed success fraction. A likelihood is not automatically a probability distribution over the parameter: its integral over parameter values need not equal one. Bayesian inference adds a prior and normalizes to obtain a posterior distribution.

L(p;x_1,\ldots,x_n)=p^s(1-p)^{n-s},\qquad s=\sum_{i=1}^{n}x_i,\quad \widehat p=\frac{s}{n}

For n>0 independent Bernoulli observations with common p, this is the likelihood of the ordered observed sequence. The estimate maximizes it, including the endpoint cases s=0 or s=n.↔ Scroll wide equations horizontally.

Confidence describes a procedure's coverage

A 95% confidence-interval procedure captures the fixed parameter in 95% of repeated samples under its assumptions. Once a specific interval has been observed, it either contains that parameter or it does not. The frequentist statement is about the method's long-run coverage, not a 95% posterior probability assigned to the parameter. The distinction matters whenever two methods use different assumptions or answer different uncertainty questions.

For independent normal observations with known population standard deviation, the sample mean has a normal sampling distribution with standard deviation $\sigma/\sqrt n$ . This gives an exact normal-theory interval. If variance is unknown, replacing it by a sample estimate requires a different derivation, such as the Student t interval for independent normal data. Large-sample approximations can be useful, but dependence, extreme tails, or selection bias can invalidate an apparently narrow interval.

\overline X\sim N\!\left(\mu,\frac{\sigma^2}{n}\right),\qquad C(X)=\left[\overline X-z_{0.975}\frac{\sigma}{\sqrt n},\ \overline X+z_{0.975}\frac{\sigma}{\sqrt n}\right]

Assume independent identically distributed normal observations and known σ>0. Here z₀.₉₇₅≈1.959964, giving exact 95% coverage before rounding.↔ Scroll wide equations horizontally.

A posterior answers a conditional question

A Bayesian prior specifies parameter uncertainty before this sample. Multiplying by likelihood and normalizing produces the posterior. A credible interval can contain 95% posterior probability, conditional on those assumptions and data. This differs from a frequentist coverage guarantee. State the prior explicitly; it is an assumption rather than a neutral fact.

A uniform prior over a Bernoulli probability becomes a Beta distribution with parameters $s+1$ and $n-s+1$ after $s$ successes in $n$ trials. Its posterior mean differs from the maximum-likelihood fraction. In engineering experiments, representative sampling, independent evaluation, and avoiding repeated peeking matter alongside estimation. Quantifying model-based uncertainty does not make a biased experiment representative.

Work it through

Construct an interval with explicit assumptions

Assume 64 independent normally distributed measurements with unknown mean μ and known standard deviation 8 units. The sample mean is 102 units. Construct a 95% confidence interval for μ.

Identify the sampling distribution before inserting numbers. Under the stated assumptions, the mean has standard error $\sigma/\sqrt n=8/\sqrt{64}=1$ unit.
Use the standard normal 97.5th percentile, approximately 1.96. The two-sided margin is $1.96(1)=1.96$ units.
Center the interval on the observed mean: $[102-1.96,102+1.96]=[100.04,103.96]$ . State the interpretation in terms of the interval procedure's repeated-sampling coverage.

The result

The interval is approximately [100.04,103.96] units. The 95% guarantee depends on the normal, independence, common-mean, and known-variance assumptions; it is not a posterior probability for this fixed μ.

The living notebook

Explore the model

Interactive lab

Interactive lab · probability

Inspect a count model before estimating its parameter

Inspect the exact binomial count distribution using p and n. At fixed p, count mean and variance grow with n, while the success fraction has variance p(1−p)/n. These model bars are neither sampled confidence intervals nor a Bayesian posterior.

Exact probabilityHighlighted outcomeMean np

Mean · np: 5
Variance · np(1−p): 2.5
Total probability: 1

Success probability p0.5Trials n10

t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forThe sampling distribution of an estimator must be derived from a model before uncertainty can be attached to an observed estimate.

Pause & reason

Check your understanding

Interactive checks need JavaScript. You can still read the explanation.

Read the checkpoint explanation

Confidence is a repeated-sampling property of the interval rule. A posterior probability needs a Bayesian model, and a prediction interval for future observations is a different construction.

Your turn

Try it yourself

Under independent normal sampling with known σ=6, a sample of 36 observations has mean 20. Find the approximate 95% confidence interval and the margin if the sample size increases to 144 with the same mean.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

The standard error divides by the square root of n, so quadrupling n halves the margin.

Compare with the solution

For $n=36$ , the standard error is $6/6=1$ and the interval is approximately $[18.04,21.96]$ . For $n=144$ , standard error is $6/12=0.5$ , so the margin is $0.98$ . More data narrows sampling uncertainty; it does not remove systematic bias.

Keep the idea

Take it with you

An estimator is a random rule before sampling; an estimate is its observed output.
A confidence interval describes repeated-sampling coverage, while a credible interval uses posterior probability.
Sampling assumptions and study design determine whether numerical uncertainty statements are trustworthy.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.