\medskip We have seen in Section NLT that nilpotent linear transformations are almost never diagonalizable (Theorem DNLT), yet have matrix representations that are very nearly diagonal (Theorem CFNLT). Our goal in this section, and the next (Section JCF), is to obtain a matrix representation of

Suppose that $\ltdefn{T}{V}{V}$ is a linear transformation and $W$ is a subspace of $V$. Suppose further that $\lt{T}{\vect{w}}\in W$ for every $\vect{w}\in W$. Then $W$ is an **invariant subspace** of $V$ relative to $T$.

We do not have any special notation for an invariant subspace, so it is important to recognize that an invariant subspace is always relative to both a superspace ($V$) and a linear transformation ($T$), which will sometimes not be mentioned, yet will be clear from the context. Note also that the linear transformation involved must have an equal domain and codomain — the definition would not make much sense if our outputs were not of the same type as our inputs.

As usual, we begin with an example that demonstrates the existence of invariant subspaces. We will return later to understand how this example was constructed, but for now, just understand how we check the existence of the invariant subspaces.

That the trivial subspace is always an invariant subspace is a special case of the next theorem. As an easy exercise before reading the next theorem, prove that the kernel of a linear transformation (Definition KLT), $\krn{T}$, is an invariant subspace. We'll wait.

Here's one more example of invariant subspaces we have encountered previously.

Suppose that $\ltdefn{T}{V}{V}$ is a linear transformation. Suppose further that for $\vect{x}\neq\zerovector$, $\lt{\left(T-\lambda I_V\right)^k}{\vect{x}}=\zerovector$ for some $k>0$. Then $\vect{x}$ is a **generalized eigenvector** of $T$ with eigenvalue $\lambda$.

Suppose that $\ltdefn{T}{V}{V}$ is a linear transformation. Define the **generalized eigenspace** of $T$ for $\lambda$ as
\begin{align*}
\geneigenspace{T}{\lambda}
&=\setparts{\vect{x}}{\lt{\left(T-\lambda I_V\right)^k}{\vect{x}}=\zerovector\text{\ for some\ }k\geq 0}
\end{align*}
$\geneigenspace{T}{\lambda}$

So the generalized eigenspace is composed of generalized eigenvectors, plus the zero vector. As the name implies, the generalized eigenspace is a subspace of $V$. But more topically, it is an invariant subspace of $V$ relative to $T$.

Suppose that $\ltdefn{T}{V}{V}$ is a linear transformation, and $U$ is an invariant subspace of $V$ relative to $T$. Define the **restriction** of $T$ to $U$ by
\begin{align*}
\ltdefn{\restrict{T}{U}}{U}{U}&
&
\lt{\restrict{T}{U}}{\vect{u}}&=\lt{T}{\vect{u}}
\end{align*}
$\restrict{T}{U}$

It might appear that this definition has not accomplished anything, as $\restrict{T}{U}$ would appear to take on exactly the same values as $T$. And this is true. However, $\restrict{T}{U}$ differs from $T$ in the choice of domain and codomain. We tend to give little attention to the domain and codomain of functions, while their defining rules get the spotlight. But the restriction of a linear transformation is all about the choice of domain and codomain. We are

Our real interest is in the matrix representation of a linear transformation when the domain decomposes as a direct sum of invariant subspaces. Consider forming a basis $B$ of $V$ as the union of bases $B_i$ from the individual $U_i$, i.e. $B=\cup_{i=1}^m\,B_i$. Now form the matrix representation of $T$ relative to $B$. The result will be block diagonal, where each block is the matrix representation of a restriction $\restrict{T}{U_i}$ relative to a basis $B_i$, $\matrixrep{\restrict{T}{U_i}}{B_i}{B_i}$. Though we did not have the definitions to describe it then, this is exactly what was going on in the latter portion of the proof of Theorem CFNLT. Two examples should help to clarify these ideas.

Diagonalizing a linear transformation is the most extreme example of decomposing a vector space into invariant subspaces. When a linear transformation is diagonalizable, then there is a basis composed of eigenvectors (Theorem DC). Each of these basis vectors can be used individually as the lone element of a spanning set for an invariant subspace (Theorem EIS). So the domain decomposes into a direct sum of one-dimensional invariant subspaces (Theorem DSFB). The corresponding matrix representation is then block diagonal with all the blocks of size 1, i.e. the matrix is diagonal. Section NLT, Section IS and Section JCF are all devoted to generalizing this extreme situation when there are not enough eigenvectors available to make such a complete decomposition and arrive at such an elegant matrix representation.

One last theorem will roll up much of this section and Section NLT into one nice, neat package.

Suppose $\ltdefn{T}{V}{V}$ is a linear transformation with eigenvalue $\lambda$. Then the **index** of $\lambda$, $\indx{T}{\lambda}$, is the index of the nilpotent linear transformation $\restrict{T}{\geneigenspace{T}{\lambda}}-\lambda I_{\geneigenspace{T}{\lambda}}$.
$\indx{T}{\lambda}$