Statistical models and identifiability

Write the data as a random element $X$ in a specified measurable sample space. A model is a collection of probability laws $\{P_\theta:\theta\in\Theta\}$ on that space. The observed value is lowercase $x$; the parameter indexes a candidate law and is fixed but unknown in the frequentist formulation used here. A statistic is a measurable function of the data that does not require the unknown parameter. An estimand $g(\theta)$ is the particular feature we hope to learn, which need not be the entire parameter.

For independent binary observations sharing a probability $p$, the sample space is all length-$n$ zero-one sequences and the law is a product of Bernoulli probabilities. Independence and the common probability are substantive assumptions. Recording repeated reports from the same underlying event does not create independent trials. A count-only experiment has a smaller sample space than an ordered-sequence experiment; these can be equally informative about $p$ while answering different questions about time trends. State the experiment, rather than inferring it from a convenient formula.

A parameterization is identifiable if equal observable laws imply equal parameter values. This is a population statement about the entire distribution, not a promise that one finite dataset uniquely determines the parameter. Bernoulli probability is identifiable because equality of laws implies equality of the probability of the event $X_1=1$. Nevertheless, a finite record containing three successes can occur under many different probabilities. Identifiability permits eventual learning under suitable sampling conditions; it does not erase sampling variation.

Consider $Y\sim N(\theta^2,1)$ with $\theta\in\mathbb R$. Values $\theta$ and $-\theta$ give exactly the same law, so sign is not identifiable. The quantity $\theta^2$ is identifiable because a normal law's mean determines it. Restricting the parameter space to nonnegative values removes the sign ambiguity, but that restriction must come from a scientific definition or an explicit modeling convention. A numerical optimizer selecting the positive solution has not supplied new evidence about sign.

An unidentifiable quantity cannot have an estimator that is consistent at two observationally indistinguishable parameter points with different target values. Under either point the estimator has the same distribution for every sample size. It therefore cannot converge in probability to two different constants: disjoint neighborhoods of those constants would both need probability approaching one under a single law. This argument needs no optimizer, information matrix, or asymptotic normal approximation. It is a direct obstruction created by the experiment.

There are several constructive responses. Report an identifiable function, impose a justified constraint, or collect an additional measurement that breaks the symmetry. In a mixture model, relabeling components may preserve the distribution, so the unordered component collection can be meaningful even when labeled components are not. Separately, a wrong model can be identifiable internally and still give misleading estimates. Model diagnostics examine compatibility with observed patterns; no finite goodness-of-fit check proves that every assumption holds. Keep structural identifiability, sampling uncertainty, and model adequacy as separate entries in your analysis.