Likelihood and sufficient statistics

For a model with densities or probability masses $f_\theta$ relative to a common dominating measure, the likelihood is $L(\theta;x)=f_\theta(x)$ viewed as a function of $\theta$. It need not sum or integrate to one over the parameter space. A density value is not the probability of a continuously distributed observation taking one exact value. Multiplying likelihood by a positive factor depending only on the data preserves all likelihood ratios and maximizers, but multiplying by a parameter-dependent factor generally changes the inference.

If an ordered Bernoulli record has $k$ successes, its likelihood is $p^k(1-p)^{n-k}$. If only the count was recorded, its likelihood also contains the binomial coefficient. This factor counts compatible sequences and is constant as a function of $p$. The two experiments consequently yield proportional likelihoods for the common probability. They do not have identical sample spaces, and proportional likelihood alone should not be confused with equality of all possible frequentist procedures under arbitrary sampling plans.

A statistic $T$ is sufficient when a version of the conditional distribution of the complete data given $T$ can be chosen independent of the parameter throughout the model. In a finite Bernoulli sample, conditioning on $T=k$ makes all sequences with that total equally likely. The unknown factors cancel between numerator and denominator. Once the total is known, the remaining ordering uncertainty carries no additional information about the common probability according to this model.

For dominated models, the factorization theorem provides a practical equivalent: the density can be written as a nonnegative parameter-dependent function of $T(x)$ times a nonnegative data-only function. Supports and zero-density cases must be represented correctly. For a uniform distribution on $[0,\theta]$, the sample maximum enters through an indicator constraining $\theta$; discarding that indicator destroys the likelihood. Sufficiency is always relative to the selected family, so adding a time-varying success parameter can make a previously sufficient total insufficient.

Sufficiency need not mean minimality. The full dataset is trivially sufficient, and appending irrelevant information to a sufficient statistic keeps it sufficient. A minimal sufficient statistic compresses as far as possible in the sense that it is a measurable function of every sufficient statistic, with the customary almost-sure qualifications. In finite positive-density models, comparing which sample likelihood ratios are constant in the parameter often identifies the minimal partition. For Bernoulli data with interior probability, two records lie in the same partition exactly when they have the same total.

Keep raw observations when checking assumptions. A record of ten zeros followed by ten ones and an alternating record have equal likelihoods for a single common Bernoulli probability, yet suggest different temporal mechanisms. Sufficiency says their orders do not help estimate that one parameter if independence and identical distribution hold. It does not certify those assumptions or erase the value of provenance. The practical workflow is to preserve the original data for diagnostics while using sufficient summaries for calculations justified by the declared model.