Hilbert spaces and orthogonal projection

An inner product measures lengths and angles through a positive definite pairing. Throughout this module the complex inner product is conjugate linear in its first argument and linear in its second. Its induced norm obeys the parallelogram identity, a stronger geometric property than the triangle inequality. A Hilbert space is complete in this norm: every Cauchy sequence has a limit inside the space. Infinite-dimensional examples include square-summable sequences and square-integrable functions, with the latter understood as almost-everywhere equivalence classes.

The completion is consequential. Polynomials on [0,1], equipped with the L2 norm, form an inner-product space but are not complete. They can approximate nonpolynomial functions in mean square. Passing to L2 adds the missing limits without assigning unique values at individual points. When a calculation uses point evaluation, derivatives, or boundary traces, further regularity must be established. A norm that controls integrated squared error does not by itself control the largest pointwise error.

Let M be a closed linear subspace of a Hilbert space H. A sequence of candidates whose distances to x approach the infimum is Cauchy by the parallelogram identity. Completeness gives a limit in H, and closedness keeps it in M. Call it P_M x. Varying that candidate along any direction in M shows that the residual is perpendicular to M. Otherwise a small move along that direction would reduce the distance, contradicting minimality.

Orthogonality also proves uniqueness and the error formula. For any other candidate m, the residual x−P_M x and the difference P_M x−m are perpendicular, so their squared norms add. No other candidate improves the error. The resulting projection is linear, self-adjoint, idempotent, and has norm one when M is nonzero. For the zero subspace its norm is zero. Every nonempty closed convex subset of H also has a unique nearest point, but its nearest-point map need not be linear.

If e_1 through e_N are orthonormal, the projection onto their span is the sum of each vector times its inner-product coefficient. Expanding the approximation space decreases the best error. For an orthonormal basis, completeness of the basis means these projections converge in norm to every vector. This is different from merely having infinitely many mutually perpendicular vectors: their closed span could still miss an entire perpendicular component of the space.

A numerical discretization replaces a large space by a finite span, where closedness is automatic. That proves a best approximation within the chosen span, not convergence of the entire numerical method. One must also control consistency, stability, and how the trial spaces approach the intended solution class. The lab below only shows a single vector and its coordinate projections. It makes perpendicular components visible, but cannot demonstrate an arbitrary Hilbert-space projection or establish density of a function basis.