Matrix norms and condition numbers

Given a vector norm, its induced matrix norm is the supremum of the output norm divided by the input norm over nonzero vectors. This definition immediately gives the compatible inequality for multiplying a vector and the submultiplicative inequality for multiplying matrices. In finite-dimensional spaces the maximum is attained on the unit sphere. The induced one-norm is the largest absolute column sum, the induced infinity-norm is the largest absolute row sum, and the induced two-norm is the largest singular value. They answer different questions because their unit balls differ.

The Frobenius norm instead sums the squares of all entries and takes a square root. It is invariant under orthogonal changes of basis and is useful for total approximation error, but it is not the matrix norm induced by the Euclidean vector norm. For an identity matrix of order n, the induced two-norm is one and the Frobenius norm is the square root of n. Confusing these values can insert a dimension-dependent factor into a condition estimate. Always label the norm rather than relying on an unqualified double-bar symbol.

For a nonsingular square matrix, define its normwise condition number as the matrix norm times the norm of its inverse. In the two-norm this is the largest singular value divided by the smallest. A direction almost annihilated by A corresponds to a large inverse response. The condition number is at least one for induced norms and is unchanged when A is multiplied by a nonzero scalar. A small determinant alone is not a condition estimate: scaling an identity changes its determinant dramatically while leaving its condition number equal to one.

If A is fixed and only a nonzero right-hand side b changes, the solution change is exactly A inverse times the right-hand-side change. A norm bound gives relative solution change at most the matrix condition number times relative input change. This bound is a worst-case statement over directions and right-hand sides, not a prediction that every perturbation is amplified maximally. Some inputs align with insensitive directions. For a singular matrix, ordinary inversion is undefined; assigning infinite condition describes that failure but does not specify a least-squares or pseudoinverse problem.

Changing units rescales coordinates and can substantially alter a normwise condition number. This is sometimes appropriate because the original variables had incompatible scales. It is not permission to conceal a sensitive physical observable: transforming a computed solution back to its original units can restore the same practical uncertainty. Componentwise perturbation models can better describe data whose small entries deserve proportionally small uncertainty. They lead to different condition measures than allowing an arbitrary perturbation of a fixed matrix norm.

Computing an explicit inverse merely to estimate conditioning is usually unnecessary and expensive. Factorization-based estimators use triangular solves or related operations to estimate inverse norms without forming every inverse entry. A reciprocal condition estimate near zero warns that small backward errors may lead to large forward errors, but the estimate's accuracy and norm convention still matter. Combine it with the residual, scaling information, and the requested output quantity. A whole-vector worst-case condition can overstate uncertainty in one well-determined scalar component.