Conditioning and backward error

Write the mathematical problem as a map y=F(d) from data to an answer. Forward error compares a computed answer with F(d). Absolute error uses their distance; relative error divides by a chosen answer scale. When the exact answer is zero, relative error with that denominator is undefined, so an absolute tolerance or an external physical scale is necessary. The norm, units, and quantities allowed to vary are part of the statement, rather than optional reporting details.

For a differentiable scalar map with nonzero input and output, a local relative condition number follows from the derivative. It predicts amplification for sufficiently small input perturbations, not a universal bound for arbitrary changes. A large value signals sensitivity inherent in the map. For example, subtracting a fixed reference from a nearly equal measured value can magnify relative uncertainty enormously, even when the subtraction itself is performed exactly. More arithmetic precision cannot recover measurement digits that were never supplied.

Backward error asks whether the computed answer is the exact solution for slightly changed data. This reverses the usual comparison: instead of moving the answer, move the inputs. An algorithm is backward stable when those changes can be bounded at roughly the arithmetic rounding level, with appropriate problem-size factors. The allowed changes matter. Perturbing every matrix entry independently is a different claim from perturbing only a measured right-hand side or preserving a matrix's symmetry.

For a linear system, the residual is b minus A times the computed vector. It is precisely a right-hand-side perturbation that makes that vector exact while A stays fixed, up to sign convention. A small unscaled residual is not automatically impressive: multiplying every equation by a huge or tiny common factor changes its size without changing the solution. Normalize using meaningful data scales, and check whether residual evaluation itself needs higher precision when cancellation is severe.

For an invertible matrix and a compatible operator norm, A inverse converts residual into solution error. Its norm may be enormous in a direction that the residual happens to occupy. The product of the matrix norm and inverse norm is the matrix condition number. Thus a tiny backward error and a large condition number can coexist with substantial forward error. This is a consequence of the geometry of nearly collapsed directions, not a contradiction in the error definitions.

A useful report includes the computed quantity, a scaled residual, an estimate of conditioning, and the uncertainty already present in the data. If the requested accuracy is unsupported, reformulate the observable, collect better data, regularize with an explicitly stated modeling assumption, or increase precision when arithmetic error dominates. Merely requesting a tighter solver tolerance may spend time without improving trustworthy digits. Residual checks and refinement help diagnose the computation; they do not change the information content of an ill-conditioned problem.