The discrepancy principle and parameter choice

Without a source-rate assumption, a sufficient deterministic convergence rule is α(δ)→0 together with δ/√α(δ)→0 as δ→0. The first limit removes bias for each fixed minimum-norm exact solution; the second controls worst-case noise amplification. Neither limit alone is sufficient to establish the usual convergence conclusion. Keeping α fixed preserves stability but may retain bias, whereas decreasing it too quickly can amplify a shrinking data error into a nonshrinking object error.

For example, α=δ satisfies both conditions, while α=δ² makes the displayed noise bound remain of constant size. That failure of a sufficient condition does not assert divergence for every special datum, but it prevents this general guarantee. An optimal rate-based choice also needs a source exponent and source bound, often unknown in practice. This motivates a posteriori rules that use the observed residual, though their guarantees still depend on accurate noise information and the properties of the chosen regularization family.

For Tikhonov, each observed singular component of the residual is multiplied by α/(σ²+α). Its magnitude increases with α. Components orthogonal to the range closure are unchanged because no object can fit them. Consequently the residual norm is continuous and nondecreasing, with lower limit the distance from yδ to the range closure and upper limit ||yδ|| as α grows. If A*yδ is nonzero, the nontrivial component makes the growth strict. A finite target must lie between those limits to admit a positive solution.

Under a valid noise bound and exact data in the range, the irreducible residual is at most δ, so a target τδ with τ>1 lies above it. If ||yδ|| is already at most τδ, the zero object already meets the stopping tolerance and one should state that convention rather than force a positive-parameter equality with no solution. Otherwise an interior target determines a positive parameter. The residual equation can be solved by monotone search, but that numerical root finding is separate from the inverse problem's stability analysis.

A discrepancy rule can deliver convergent regularization under the appropriate linear assumptions, but its object-error rate is not universal. Rate claims require source conditions and attention to the method's qualification; an a priori rate formula cannot simply be copied onto every discrepancy rule. A residual of size δ does not imply object error of size δ when the inverse is ill conditioned. It means the fit is commensurate with the stated data uncertainty, which is a different and useful conclusion.

Underestimating δ can force the algorithm to reproduce noise or model discrepancy; overestimating it can stop reconstruction before useful information has been extracted. Correlated statistical errors call for a justified weighted norm or a probabilistic tolerance, not substitution of an unexplained standard deviation into a deterministic theorem. L-curves, generalized cross-validation, and other heuristics can be useful choices, but they rely on different assumptions and do not acquire the discrepancy principle's guarantees merely by producing a visually appealing parameter.