Complete reducibility of infinite-dimensional representations

Question

We have Maschke’s theorem stated in the following form (I am working over a field $k$ of characteristic $0$):

Theorem. Let $G$ be a finite group (or a compact Lie group), and let $V$ be a (possibly infinite-dimensional) $G$-representation. Then for every $G$-invariant subspace $W \subseteq V$, there exists a $G$-invariant complement $U$, i.e. $V = W \oplus U$.

My aim is to prove further that every such $G$-representation is completely reducible in the sense that there exists a collection $\{V_i\}_{i \in I}$ of irreducible subrepresentations of $V$ such that $V = \bigoplus_{i \in I} V_i$. For finite-dimensional representations, one can use induction on $\dim V$, but this approach does not work for infinite-dimensional representations.

For a finite group $G$, we can construct its group algebra $k[G]$ and then use module theory, since $V$ becomes a $k[G]$-module. In module theory, it is known that a module $M$ is completely reducible (in my sense) if and only if every submodule of $M$ has a complement in $M$.

My question is: can I do something similar if $G$ is a compact Lie group? Do I have to consider $U(\mathfrak{g})$, the universal enveloping algebra of the Lie algebra of $G$?

Answer 1

Maschke's theorem in the generality you've stated it is false. For compact Lie groups and infinite-dimensional representations we need to be more careful. Here is a result:

Theorem: Let $G$ be any infinite group and $K$ be any field. Then there exists a representation $V$ of $G$ over $K$ which is not semisimple (in the sense of being a direct sum of irreducibles).

Proof. We'll take $V$ to be the regular representation of $G$ on the group algebra $K[G]$. There is a $G$-equivariant map $\varepsilon : K[G] \to K$ from the regular representation to the trivial representation sending every $g \in G$ to $1 \in K$, and in order for $K[G]$ to be semisimple, this map must split. A splitting of this map is determined by an element $r \in K[G]$ satisfying

• $\varepsilon(r) = 1$
• $r$ is invariant under the action of $G$

but this is impossible if $G$ is infinite; the first condition forces $r$ to be nonzero, but since $r$ is necessarily a finite sum $r = \sum_{g \in G} r_g g$ over finitely many elements of $G$, the coefficients $r_g$ cannot all be the same (since all but finitely many of them must be zero). $\Box$

(This result can be slightly strengthened to a precise converse of Maschke's theorem: namely, that the group algebra $K[G]$ is semisimple if and only if $G$ is finite and $\text{char}(K) \nmid |G|$.)

To get a version of complete reducibility for the case of compact Lie groups $G$ requires that we work specifically with unitary representations on Hilbert spaces $H$. In this setting we can show that if $U \subseteq H$ is a closed invariant subspace then its orthogonal complement is also closed and invariant, and we can write $H = U \oplus U^{\perp}$ as a Hilbert space direct sum. This also works if $G$ is only assumed to be a topological group as long as the representation is still unitary. However, complete reducibility does not follow because e.g. it is still false in the noncompact case, such as when $G = \mathbb{R}$ (the regular representation on $L^2(\mathbb{R})$ has no irreducible subspaces).

Complete reducibility does still hold if $G$ is compact (not necessarily a Lie group) but requires a harder argument (that I am not familiar with); this is part of the Peter-Weyl theorem.