Hierarchical models and partial pooling

Let a group estimate $z_j$ have known sampling variance $s_j^2$, with $z_j\mid\theta_j\sim N(\theta_j,s_j^2)$. Assign conditionally independent group parameters $\theta_j\mid\mu,\tau\sim N(\mu,\tau^2)$. Given the hyperparameters, groups have the same prior distribution before their data distinguish them. This is a form of conditional exchangeability, not a claim that their observed values are identical. Relevant group predictors can be introduced into the prior mean rather than pretending dissimilar groups are interchangeable.

The sampling variance measures uncertainty in a group estimate; the population variance measures heterogeneity between true group effects. Interchanging them changes the model. Known sampling variances make this lesson's conditional calculation simple, but estimating them from small samples may require an additional model. Correlation between group estimates also needs representation. Sharing one source of measurement bias is different from independent observation noise, and a hierarchical prior on group effects does not automatically repair a misspecified sampling covariance.

Multiply the normal likelihood and normal prior for one group. The coefficient of the squared parameter is the sum of sampling precision $1/s_j^2$ and prior precision $1/\tau^2$. The linear coefficient is the corresponding precision-weighted sum of $z_j$ and $\mu$. Completing the square gives a normal conditional posterior. Its mean is a convex combination: a precise group estimate receives a larger data weight, while a noisy estimate moves farther toward the population mean.

As $s_j^2$ approaches zero at fixed positive population variance, the data weight approaches one. As the population variance approaches zero at fixed sampling variance, the conditional mean approaches the common mean. These are limiting calculations, not permission to substitute zero into a density requiring positive variance. A large departure from the population mean does not, by itself, change the shrinkage weight in this fixed-variance normal model. Robust or adaptive alternatives can behave differently because they use different distributions or infer additional parameters.

In a full Bayesian hierarchy, the population mean and scale receive proper hyperpriors, and their posterior uncertainty is integrated rather than replaced by one fitted pair. For a group parameter with finite posterior second moment, the law of total variance separates average conditional variance from variance of the conditional mean across hyperparameter draws. A plug-in calculation omits this latter component and also replaces the former by its value at one point. It need not reproduce the full posterior uncertainty or even its mean.

Population-scale priors deserve predictive checks because the number of groups may be small. A wide-looking scale prior can imply absurdly heterogeneous groups, while a prior concentrated near zero can impose nearly complete pooling. Predicting a new observation in an existing group and predicting a previously unseen group also involve different integrations: the latter requires a fresh group effect from the population distribution. Computational reparameterizations can improve sampling without changing this generative model, provided the transformed density and any necessary Jacobian are handled consistently.