Likelihood-ratio tests and pivots
Separate a likelihood-ratio construction from its calibration and distinguish exact pivots from Wilks approximations.
On this page
The question for today
What you’ll understand
- Compute constrained and unconstrained likelihood maxima.
- Distinguish exact pivotal calibration from asymptotic chi-square calibration.
- Identify the regularity and smooth-constraint assumptions behind a Wilks limit.
Likelihood-ratio tests compare the best fit allowed by a null hypothesis with the best fit in the full model. The resulting statistic is an ordering of evidence, not yet a calibrated probability. Calibration requires its distribution under the null, either exactly or through a justified approximation. In a normal mean problem, an exact pivot supplies the answer. In many regular models a chi-square limit is available, but a small sample, boundary, or unidentified nuisance parameter can change the result substantially. Keep the construction and its calibration as two separate steps.
Compare the best null fit with the best overall fit
For a null subset , define as the supremum likelihood under the null divided by the supremum over the full model. This convention makes the ratio at most one when both suprema are positive and finite. Large evidence against the null corresponds to a small ratio or a large statistic . Nuisance parameters must be optimized within each parameter set rather than silently fixed at convenient values. Numerical failure to find the maxima is a computational concern distinct from statistical calibration.
For normal observations with known variance and an unknown mean, the unrestricted maximizing mean is the sample average. Under the point null the mean is fixed. Expanding the sums of squared residuals cancels the within-sample variation and leaves the squared distance between sample mean and null mean, scaled by its variance. In this special model the likelihood-ratio statistic is exactly the square of a standard-normal variable under the null. The chi-square distribution here is exact, not an application of an asymptotic theorem.
A pivot has a parameter-free distribution
A pivot is a function of data and parameter whose distribution does not depend on unknown parameters in the model. It need not be a statistic, because it may include the unknown parameter. The standardized normal sample mean with known variance is a pivot; replacing the standard deviation by the sample standard deviation yields the Student pivot under iid normal sampling, with degrees of freedom for . Their parameter-free reference laws support exact test and interval calibration.
Exact pivotal behavior depends on the distributional assumptions. The Student expression need not have a Student distribution under arbitrary heavy-tailed or dependent sampling. An asymptotically pivotal statistic instead approaches a parameter-free reference distribution after taking a specified limit. This can be useful, but its calibration error at a chosen sample size remains a question. A simulation study at several parameter values can investigate that error for those values; it does not prove uniform validity across an unrestricted model.
Why Wilks gives a limit, and where it stops
One sufficient route to Wilks's conclusion is an iid correctly specified finite-dimensional model with an interior identifiable true parameter, nonsingular finite information, a locally uniform quadratic likelihood expansion with a normal limiting score, and consistent constrained and unconstrained maximizers governed by that local expansion. Assume the null is locally a smooth embedded submanifold through the truth, with full-rank constraint derivative and fixed codimension . The quadratic approximation turns the fitted-likelihood difference into the squared information-orthogonal normal component of the limiting score, yielding a chi-square limit with degrees of freedom. This describes the proof mechanism, not a check-free recipe.
The conclusion is convergence in distribution as sample size grows at a fixed regular null parameter. It is not an exact finite-sample law. A boundary null produces a different local geometry; unidentified mixture components, singular information, or growing parameter dimension also escape the stated assumptions. Even a regular Bernoulli model can have poor small-sample chi-square calibration because its counts are discrete. When feasible, enumerate an exact null distribution or use a separately justified calibration method instead of treating the asymptotic label as a finite-sample guarantee.
Work it through
An exact likelihood-ratio test
Nine iid observations have mean . Test against an unrestricted mean using a two-sided 5% likelihood-ratio rule.
The unrestricted maximizing mean is 1.2, while the null mean is zero. The sample-mean standard deviation under either mean is 2/3.
The standardized displacement is . The likelihood-ratio statistic is its square, .
Under the null, has exactly the law. Its 95th percentile is about , so does not exceed the rejection threshold.
Do not reject at the 5% level. The two-sided p-value is approximately 0.07186; nonrejection does not establish that the true mean equals zero.
The living notebook
Explore the model
Interactive lab · probability
Audit a small-sample chi-square cutoff
Reset and set both . Predict the total exact mass in counts zero through two and eight through ten: . Sum those bars on paper. They are the rejection counts for the Bernoulli likelihood-ratio cutoff in the practice problem. The diagram gives an exact null audit; it does not claim that binomial counts themselves follow a chi-square law.
- Mean · np
- 5
- Variance · np(1−p)
- 2.5
- Total probability
- 1
These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.
Look forAn asymptotic 5% cutoff can have a substantially different exact small-sample rejection probability.
Pause & reason
Check your understanding
Your turn
Try it yourself
Ten Bernoulli trials give eight successes. For null , compute . A cutoff of rejects counts at most two or at least eight. Compute this rule's exact size under the null.
Take a moment to make a prediction or work it out on paper. Open a hint when you need one.
Give me a hint
Use symmetry of the binomial null law and sum the probabilities of counts zero, one, and two before doubling.
Compare with the solution
The observed statistic is approximately , just above the asymptotic cutoff. Exact null size is . Thus this nominal asymptotic 5% rule rejects over 10% of the time at this finite sample size. This does not contradict a limit as sample size increases at fixed interior probability.
Keep the idea
Take it with you
- A likelihood ratio defines an ordering; a null distribution supplies calibration.
- Exact pivots yield finite-sample reference laws under their model assumptions.
- Wilks requires regular local geometry and provides an asymptotic chi-square law.
Read further
Original sources and deeper explanations to follow at your own pace.
Optional progress, saved in this browser when available. Revisit any lesson at any time.
Lesson not marked complete.