Mutual information and relative entropy

Quantum relative entropy compares a state rho with a reference sigma using the trace of rho times the difference of their logarithms. Its finite value requires the support of rho to lie inside the support of sigma. Otherwise it is positive infinity: the reference assigns zero support to an event the actual state permits. Zero eigenvalues outside the support cause no contribution. Diagonal states reduce to the classical sum of probability times the logarithm of a probability ratio.

The divergence is nonnegative and vanishes exactly when the normalized states agree. It is not symmetric and is not a distance metric, so there is no general triangle inequality to invoke. Its operational significance concerns distinguishability in repeated asymmetric hypothesis tests, with carefully specified error criteria. For elementary calculations, first check supports and a common eigenbasis. If the matrices do not commute, separately diagonalize them to define their logarithms; taking logarithms of individual matrix entries is invalid.

For declared factors A and B, take the joint state and compare it with the tensor product of its partial traces. Expanding the logarithm of a tensor product gives the sum of the local logarithms on their respective factors. The partial-trace identity then turns the two local expectation terms into local entropies. This proves that mutual information equals the sum of the marginal entropies minus the joint entropy and is nonnegative.

A fair classical mixture of zero-zero and one-one has local entropies one bit and joint entropy one bit, hence mutual information one bit. A Bell pair has the same marginals but joint entropy zero, hence mutual information two bits. The latter does not mean that measuring one qubit releases two ordinary classical bits. Quantum mutual information counts total correlation before a measurement and can exceed the classical bound set by either marginal entropy.

The data-processing theorem says that applying the same completely positive, trace-preserving map to the two arguments cannot increase relative entropy. Applying a channel locally on B therefore cannot increase mutual information between A and the channel output. The comparison product remains the product of the new marginals because the channel is local. A joint operation acting on both factors can create correlations, so the word local is essential to this conclusion.

Dephasing a Bell pair in the computational basis removes its off-diagonal terms and leaves the classically correlated mixture. Its mutual information drops from two bits to one, while its local states remain unchanged. Conditional entropy, defined as joint minus conditioning-system entropy, correspondingly changes from negative one bit to zero. Negative quantum conditional entropy is allowed. Postselecting on an announced outcome defines a different conditional task; a selected branch need not satisfy the unconditioned comparison without accounting for its classical record.