Invariant subspaces and Maschke’s theorem

A nonzero representation is irreducible if its only invariant subspaces are zero and the whole space. A representation is completely reducible, or semisimple, if it is a direct sum of irreducible subrepresentations. These definitions differ: a representation may contain an irreducible subspace without splitting around it. Finite dimensionality makes repeated splitting terminate once complements exist. Induct on dimension, splitting off a proper invariant subspace whenever the current piece is not irreducible.

Let the field be arbitrary for the moment and assume its characteristic does not divide the finite group order. Choose an invariant subspace and any linear projection onto it, by extending a basis. Conjugate that projection by every representing operator and average. Invariance guarantees that every conjugated map still lands in the chosen subspace and fixes that subspace pointwise. The average therefore retains both properties; this is why it remains a projection, although averaging arbitrary idempotents would not generally preserve idempotence.

Conjugating the average by a fixed group element simply permutes its summands, so the averaged projection commutes with the whole action. Its kernel is consequently invariant. Every vector is the sum of its projected part and its difference from that projection, placing it in the sum of the chosen subspace and the kernel. Their intersection is zero because the projection is the identity on the first and zero on the second. This proves the invariant direct-sum decomposition and hence complete reducibility.

Over the complex numbers there is another useful averaging construction. Start with any positive definite Hermitian inner product and sum its translates under the action. The result is still positive definite and becomes invariant. Orthogonal complements in this new geometry are invariant, yielding the same splitting conclusion. The original coordinates need not make the matrices unitary, but an orthonormal basis for the averaged form does. This justifies using unitary matrices later without assuming that an arbitrary initial basis already has that property.

In characteristic a prime dividing the group order, division by the group order is impossible. This is a genuine obstruction rather than merely a failure of one proof. Over a field of characteristic p, let a generator of the cyclic group of order p act on a two-dimensional space by the identity plus the matrix with a single upper-right entry one. Its pth power is the identity, since the square of the added matrix is zero and the intermediate binomial coefficient p vanishes.

The first coordinate line is invariant, but no other line complements it invariantly. Any complementary line has a vector with nonzero second coordinate. Applying the nilpotent part sends that vector to a nonzero vector in the first coordinate line, which cannot lie in the complementary line. Thus the action is reducible and indecomposable. The characteristic hypothesis guarantees splitting for all representations; its failure does not mean every particular representation must fail to split. Trivial representations still decompose into trivial lines.