core30 min studyLesson 249 in Mathematics

Likelihood-ratio tests and pivots

Separate a likelihood-ratio construction from its calibration and distinguish exact pivots from Wilks approximations.

On this page

What you’ll understand

  • Compute constrained and unconstrained likelihood maxima.
  • Distinguish exact pivotal calibration from asymptotic chi-square calibration.
  • Identify the regularity and smooth-constraint assumptions behind a Wilks limit.

Likelihood-ratio tests compare the best fit allowed by a null hypothesis with the best fit in the full model. The resulting statistic is an ordering of evidence, not yet a calibrated probability. Calibration requires its distribution under the null, either exactly or through a justified approximation. In a normal mean problem, an exact pivot supplies the answer. In many regular models a chi-square limit is available, but a small sample, boundary, or unidentified nuisance parameter can change the result substantially. Keep the construction and its calibration as two separate steps.

Compare the best null fit with the best overall fit

For a null subset Θ0⊆Θ\Theta_0\subseteq\Theta, define Λ\Lambda as the supremum likelihood under the null divided by the supremum over the full model. This convention makes the ratio at most one when both suprema are positive and finite. Large evidence against the null corresponds to a small ratio or a large statistic D=−2log⁡ΛD=-2\log\Lambda. Nuisance parameters must be optimized within each parameter set rather than silently fixed at convenient values. Numerical failure to find the maxima is a computational concern distinct from statistical calibration.

For normal observations with known variance and an unknown mean, the unrestricted maximizing mean is the sample average. Under the point null the mean is fixed. Expanding the sums of squared residuals cancels the within-sample variation and leaves the squared distance between sample mean and null mean, scaled by its variance. In this special model the likelihood-ratio statistic is exactly the square of a standard-normal variable under the null. The chi-square distribution here is exact, not an application of an asymptotic theorem.

Λ(x)=sup⁡θ∈Θ0L(θ;x)sup⁡θ∈ΘL(θ;x),D=−2log⁡Λ;D=n(Xˉ−μ0)2σ2∼χ12 under the normal mean null\Lambda(x)=\frac{\sup_{\theta\in\Theta_0}L(\theta;x)}{\sup_{\theta\in\Theta}L(\theta;x)},\qquad D=-2\log\Lambda;\qquad D=\frac{n(\bar X-\mu_0)^2}{\sigma^2}\sim\chi_1^2\ \text{under the normal mean null}
The last identity assumes iid normal observations and known positive variance. Other models require their own null-distribution argument.↔ Scroll wide equations horizontally.

A pivot has a parameter-free distribution

A pivot is a function of data and parameter whose distribution does not depend on unknown parameters in the model. It need not be a statistic, because it may include the unknown parameter. The standardized normal sample mean with known variance is a pivot; replacing the standard deviation by the sample standard deviation yields the Student pivot under iid normal sampling, with n−1n-1 degrees of freedom for n≥2n\ge2. Their parameter-free reference laws support exact test and interval calibration.

Exact pivotal behavior depends on the distributional assumptions. The Student expression need not have a Student distribution under arbitrary heavy-tailed or dependent sampling. An asymptotically pivotal statistic instead approaches a parameter-free reference distribution after taking a specified limit. This can be useful, but its calibration error at a chosen sample size remains a question. A simulation study at several parameter values can investigate that error for those values; it does not prove uniform validity across an unrestricted model.

Z=n(Xˉ−μ)σ∼N(0,1),T=n(Xˉ−μ)S∼tn−1,S2=1n−1∑i=1n(Xi−Xˉ)2Z=\frac{\sqrt n(\bar X-\mu)}{\sigma}\sim N(0,1),\qquad T=\frac{\sqrt n(\bar X-\mu)}{S}\sim t_{n-1},\quad S^2=\frac1{n-1}\sum_{i=1}^n(X_i-\bar X)^2
These are exact pivots for iid normal data with positive variance. The first uses known sigma; the second accommodates unknown sigma through S.↔ Scroll wide equations horizontally.

Why Wilks gives a limit, and where it stops

One sufficient route to Wilks's conclusion is an iid correctly specified finite-dimensional model with an interior identifiable true parameter, nonsingular finite information, a locally uniform quadratic likelihood expansion with a normal limiting score, and consistent constrained and unconstrained maximizers governed by that local expansion. Assume the null is locally a smooth embedded submanifold through the truth, with full-rank constraint derivative and fixed codimension rr. The quadratic approximation turns the fitted-likelihood difference into the squared information-orthogonal normal component of the limiting score, yielding a chi-square limit with rr degrees of freedom. This describes the proof mechanism, not a check-free recipe.

The conclusion is convergence in distribution as sample size grows at a fixed regular null parameter. It is not an exact finite-sample law. A boundary null produces a different local geometry; unidentified mixture components, singular information, or growing parameter dimension also escape the stated assumptions. Even a regular Bernoulli model can have poor small-sample chi-square calibration because its counts are discrete. When feasible, enumerate an exact null distribution or use a separately justified calibration method instead of treating the asymptotic label as a finite-sample guarantee.

An exact likelihood-ratio test

Nine iid N(μ,4)N(\mu,4) observations have mean 1.21.2. Test μ=0\mu=0 against an unrestricted mean using a two-sided 5% likelihood-ratio rule.

  1. The unrestricted maximizing mean is 1.2, while the null mean is zero. The sample-mean standard deviation under either mean is 2/3.

  2. The standardized displacement is 1.2/(2/3)=1.81.2/(2/3)=1.8. The likelihood-ratio statistic is its square, D=3.24D=3.24.

  3. Under the null, DD has exactly the χ12\chi_1^2 law. Its 95th percentile is about 3.8414593.841459, so 3.243.24 does not exceed the rejection threshold.

The result

Do not reject at the 5% level. The two-sided p-value is approximately 0.07186; nonrejection does not establish that the true mean equals zero.

Explore the model

Interactive lab

Interactive lab · probability

Audit a small-sample chi-square cutoff

Reset and set both p=0.5,n=10p=0.5,n=10. Predict the total exact mass in counts zero through two and eight through ten: 112/1024=0.109375112/1024=0.109375. Sum those bars on paper. They are the rejection counts for the Bernoulli likelihood-ratio cutoff 3.8414593.841459 in the practice problem. The diagram gives an exact null audit; it does not claim that binomial counts themselves follow a chi-square law.

Audit a small-sample chi-square cutoffThese are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Current readings: Mean · np: 5; Variance · np(1−p): 2.5; Total probability: 1. All parameters are adjustable with the labeled controls below.03581000.070.150.220.3Successes kP(X = k)k = 0 · P = 9.77e-4
Exact probabilityHighlighted outcomeMean np
Mean · np
5
Variance · np(1−p)
2.5
Total probability
1
t = 0 s

These are exact binomial probabilities for independent trials with one fixed success probability. Playback tours the distribution; it does not resample outcomes. Parameter changes pause playback and recompute the model at the current elapsed time. Reset restores both parameters and returns time to zero. Playback starts paused. Adjust either parameter, then choose Play or Step.

Look forAn asymptotic 5% cutoff can have a substantially different exact small-sample rejection probability.

Check your understanding

Which statement about Wilks's chi-square result is correct under its regular fixed-dimensional hypotheses?

Try it yourself

Ten Bernoulli trials give eight successes. For null p=0.5p=0.5, compute D=2[8log⁡(1.6)+2log⁡(0.4)]D=2[8\log(1.6)+2\log(0.4)]. A cutoff of 3.8414593.841459 rejects counts at most two or at least eight. Compute this rule's exact size under the null.

Take a moment to make a prediction or work it out on paper. Open a hint when you need one.

Give me a hint

Use symmetry of the binomial null law and sum the probabilities of counts zero, one, and two before doubling.

Compare with the solution

The observed statistic is approximately 3.8548953.854895, just above the asymptotic cutoff. Exact null size is 2[(100)+(101)+(102)]/210=2(1+10+45)/1024=0.1093752[\binom{10}0+\binom{10}1+\binom{10}2]/2^{10}=2(1+10+45)/1024=0.109375. Thus this nominal asymptotic 5% rule rejects over 10% of the time at this finite sample size. This does not contradict a limit as sample size increases at fixed interior probability.

Take it with you

  • A likelihood ratio defines an ordering; a null distribution supplies calibration.
  • Exact pivots yield finite-sample reference laws under their model assumptions.
  • Wilks requires regular local geometry and provides an asymptotic chi-square law.

Read further

Original sources and deeper explanations to follow at your own pace.

Optional progress, saved in this browser when available. Revisit any lesson at any time.

Lesson not marked complete.