Gaussian channels and water filling

Consider Y=X+Z with independent zero-mean Gaussian noise of variance N>0. Uses are memoryless and the block codewords satisfy squared norm at most nP, where P≥0 is the average power per real use. The corresponding single-use information maximization allows E X²≤P. Differential entropy of the output is bounded by that of a Gaussian with the same variance, while the conditional output entropy equals the entropy of the independent noise.

A zero-mean Gaussian input of variance P attains the information maximum, yielding half log₂ of one plus signal-to-noise ratio. The operational coding theorem can enforce the block power constraint through suitable long code constructions rather than assuming every iid Gaussian draw already obeys it. A complex use contains two real dimensions under its own total noise and power convention; importing its formula without translating units can silently double a claimed real-use rate.

Now one vector use contains m independent real Gaussian noise modes with strictly positive variances N_i. Allocate powers P_i≥0 with their sum at most one total budget P. Independent Gaussian input components maximize information for a chosen allocation, and the rates add across modes. The resulting total is bits per vector use, not per scalar component. Dividing by m would be a separate normalization, while a time or bandwidth rate needs an additional physical sampling convention.

The objective is concave in the powers. For active modes, the derivative is proportional to one divided by N_i+P_i, so the constrained optimum equalizes those total levels. Inactive modes have noise already above the common level. The level is chosen so the allocated powers sum to P when P is positive. This Karush–Kuhn–Tucker argument also explains why a mode begins receiving power only when the water level reaches its noise floor.

Differential entropy can be negative and changes under a nonsingular linear coordinate map by the logarithm of its absolute determinant. That does not contradict information invariance: applying a bijective measurable change to one observed variable preserves its mutual information with another. In the Gaussian proof, coordinate-dependent differential entropies occur in a difference whose scale terms cancel. Treating differential entropy alone as a count of distinguishable messages loses that cancellation.

The water-filling result assumes the stated independent additive Gaussian modes and quadratic sum constraint. Correlated noise can be diagonalized when the available transformations and input constraint justify doing so. Peak-amplitude restrictions, non-Gaussian noise or a different cost can change the optimizing distribution and allocation. A channel designer must therefore identify the allowed signal set and noise law before interpreting a logarithmic expression as an achievable capacity.