Decision theory and minimax risk

A decision rule $\delta(X)$ maps observations to an action. A loss $L(\theta,a)$ measures the cost of action $a$ if the parameter is $\theta$, and risk averages that loss under the sampling law at the fixed parameter. Squared-error risk combines bias and variance; asymmetric costs or classification decisions lead to other losses. Changing the loss changes the optimization problem, so an optimal estimator must always be described together with its model, target, action space, and loss.

An estimator dominates another if its risk is no larger everywhere and smaller somewhere. When risk curves cross, dominance cannot rank them. Bayes analysis averages risk against a specified proper prior and minimizes that average. Minimax analysis minimizes the supremum of risk over the parameter space. The prior can be used as a mathematical device to obtain a lower bound on worst-case performance even if no subjective interpretation of it is intended. Neither criterion excuses checking whether the model and loss represent the actual application.

Let $K\sim\operatorname{Binomial}(n,p)$ with $p\in[0,1]$, and estimate $p$ under squared error. The sample proportion has risk $p(1-p)/n$, maximized at the center. Consider instead $\delta_a=(K+a)/(n+2a)$ for $a>0$. Its mean is drawn toward one half. Its bias is $a(1-2p)/(n+2a)$, and its variance is $np(1-p)/(n+2a)^2$. Adding their squared-bias and variance contributions gives a risk polynomial that can be inspected exactly.

Set $a=\sqrt n/2$. Since $(1-2p)^2=1-4p(1-p)$, the coefficient of $p(1-p)$ in the numerator cancels, leaving risk constant in $p$. This rule sacrifices zero risk at the endpoints to reduce the largest risk near the center. Constant risk alone does not prove minimaxity: there could still exist a different rule with a lower constant or lower worst case. A lower bound matching this achieved constant is the missing ingredient.

Use the proper beta prior with equal positive shape parameters $a$. Multiplying its density by the binomial likelihood gives a beta posterior with shapes $K+a$ and $n-K+a$. Under squared-error loss the posterior mean minimizes posterior expected loss, because completing the square separates posterior variance from squared distance to that mean. Hence $\delta_a$ is Bayes for this prior. At the selected value of $a$, its constant risk is also its integrated Bayes risk.

Every competing rule has worst-case risk at least its prior-averaged risk, which is at least the Bayes optimum. The candidate attains exactly that value at every parameter, so the lower and upper bounds coincide and prove minimaxity. This proof covers arbitrary measurable rules with well-defined risks, including randomized rules: conditional averaging cannot increase squared-error loss. It establishes the criterion for this binomial experiment and squared-error target only. A different parameter restriction or loss can select a different procedure, and minimaxity does not mean uniform dominance over the unbiased sample proportion.