Section IS  Invariant Subspaces

From A First Course in Linear Algebra
Version 1.04
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/

Draft: This Section Complete, But Subject To Change

We have seen in Section NLT that nilpotent linear transformations are almost never diagonalizable (Theorem DNLT), yet have matrix representations that are very nearly diagonal (Theorem CFNLT). Our goal in this section, and the next (Section JCF), is to obtain a matrix representation of any linear transformation that is very nearly diagonal. A key step in reaching this goal is an understanding of invariant subspaces, and a particular type of invariant subspace that contains vectors known as “generalized eigenvectors.”

Subsection IS: Invariant Subspaces

As is often the case, we start with a definition.

Definition IS
Invariant Subspace
Suppose that $T:V\mapsto V$ is a linear transformation and $W$ is a subspace of $V$. Suppose further that $T\left(w\right)\in W$ for every $w\in W$. Then $W$ is an invariant subspace of $V$ relative to $T$. $△$

We do not have any special notation for an invariant subspace, so it is important to recognize that an invariant subspace is always relative to both a superspace ($V$) and a linear transformation ($T$), which will sometimes not be mentioned, yet will be clear from the context. Note also that the linear transformation involved must have an equal domain and codomain — the definition would not make much sense if our outputs were not of the same type as our inputs.

As usual, we begin with an example that demonstrates the existence of invariant subspaces. We will return later to understand how this example was constructed, but for now, just understand how we check the existence of the invariant subspaces.

Example TIS
Two invariant subspaces
Consider the linear transformation $T:{ℂ}^4\mapsto {ℂ}^4$ defined by $T\left(x\right)=Ax$ where $A$ is given by

Define (with zero motivation),

and set $W=〈\left\{w_1,w_2\right\}〉$. We verify that $W$ is an invariant subspace of ${ℂ}^4$ with respect to $T$. By the definition of $W$, any vector chosen from $W$ can be written as a linear combination of $w_1$ and $w_2$. Suppose that $w\in W$, and then check the details of the following verification,

So, by Definition IS, $W$ is an invariant subspace of ${ℂ}^4$ relative to $T$. In an entirely similar manner we construct another invariant subspace of $T$.

With zero motivation, define

and set $X=〈\left\{x_1,x_2\right\}〉$. We verify that $X$ is an invariant subspace of ${ℂ}^4$ with respect to $T$. By the definition of $X$, any vector chosen from $X$ can be written as a linear combination of $x_1$ and $x_2$. Suppose that $x\in X$, and then check the details of the following verification,

So, by Definition IS, $X$ is an invariant subspace of ${ℂ}^4$ relative to $T$.

There is a bit of magic in each of these verifications where the two outputs of $T$ happen to equal linear combinations of the two inputs. But this is the essential nature of an invariant subspace. We’ll have a peek under the hood later, and it won’t look so magical after all.

As a hint of things to come, verify that $B=\left\{w_1,w_2,x_1,x_2\right\}$ is a basis of ${ℂ}^4$. Splitting this basis in half, Theorem DSFB, tells us that ${ℂ}^4=W\oplus X$. To see why a decomposition of a vector space into a direct sum of invariant subspaces might be interesting, construct the matrix representation of $T$ relative to $B$, $M_{B,B}^T$. Hmmmmmm. $⊠$

Example TIS is a bit mysterious at this stage. Do we know any other examples of invariant subspaces? Yes, as it turns out, we have already seen quite a few. We’ll give some examples now, and in more general situations, describe broad classes of invariant subspaces with theorems. First up is eigenspaces.

Theorem EIS
Eigenspaces are Invariant Subspaces
Suppose that $T:V\mapsto V$ is a linear transformation with eigenvalue $\lambda$ and associated eigenspace ${\mathcal{ℰ}}_T\left(\lambda \right)$. Let $W$ be any subspace of ${\mathcal{ℰ}}_T\left(\lambda \right)$. Then $W$ is an invariant subspace of $V$ relative to $T$. $\square$

Proof   Choose $w\in W$. Then

So by Definition IS, $W$ is an invariant subspace of $V$ relative to $T$. $■$

Theorem EIS is general enough to determine that an entire eigenspace is an invariant subspace, or that simply the span of a single eigenvector is an invariant subspace. It is not always the case that any subspace of an invariant subspace is again an invariant subspace, but eigenspaces do have this property. Here is an example of the theorem, which also allows us to very quickly build several several invariant (4x4, 2 evs, 1 2x2 jordan, 1 2x2 diag)

Example EIS
Eigenspaces as invariant subspaces
Define the linear transformation $S:M_{22}\mapsto M_{22}$ by

Build a matrix representation of $S$ relative to the standard basis (Definition MR, Example BM) and compute eigenvalues and eigenspaces of $S$ with the computational techniques of Chapter E in concert with Theorem EER. Then

So by Theorem EIS, both ${\mathcal{ℰ}}_S\left(1\right)$ and ${\mathcal{ℰ}}_S\left(2\right)$ are invariant subspaces of $M_{22}$ relative to $S$. However, Theorem EIS provides even more invariant subspaces. Since ${\mathcal{ℰ}}_S\left(1\right)$ has dimension 1, it has no interesting subspaces, however ${\mathcal{ℰ}}_S\left(2\right)$ has dimension 2 and has a plethora of subspaces. For example, set

and define $U=〈\left\{u\right\}〉$. Then since $U$ is a subspace of ${\mathcal{ℰ}}_S\left(2\right)$, Theorem EIS says that $U$ is an invariant subspace of $M_{22}$ (or we could check this claim directly based simply on the fact that $u$ is an eigenvector of $S$). $⊠$

For every linear transformation there are some obvious, trivial invariant subspaces. Suppose that $T:V\mapsto V$ is a linear transformation. Then simply because $T$ is a function (Definition LT), the subspace $V$ is an invariant subspace of $T$. In only a minor twist on this theme, the range of $T$, $\mathcal{ℛ}\left(T\right)$, is an invariant subspace of $T$ by Definition RLT. Finally, Theorem LTTZZ provides the justification for claiming that $\left\{0\right\}$ is an invariant subspace of $T$.

That the trivial subspace is always an invariant subspace is a special case of the next theorem. As an easy exercise before reading the next theorem, prove that the kernel of a linear transformation (Definition KLT), $\mathcal{K}\left(T\right)$, is an invariant subspace. We’ll wait.

Theorem KPIS
Kernels of Powers are Invariant Subspaces
Suppose that $T:V\mapsto V$ is a linear transformation. Then $\mathcal{K}\left(T^k\right)$ is an invariant subspace of $V$. $\square$

Proof   Suppose that $z\in \mathcal{K}\left(T^k\right)$. Then

So by Definition KLT, we see that $T\left(z\right)\in \mathcal{K}\left(T^k\right)$. Thus $\mathcal{K}\left(T^k\right)$ is an invariant subspace of $V$ relative to $T$ (Definition IS). $■$

Two interesting special cases of Theorem KPIS occur when choose $k=0$ and $k=1$. Rather than give an example of this theorem, we will refer you back to Example KPNLT where we work with null spaces of the first four powers of a nilpotent matrix. By Theorem KPIS each of these null spaces is an invariant subspace of the associated linear transformation.

Here’s one more example of invariant subspaces we have encountered previously.

Example ISJB
Invariant subspaces and Jordan blocks
Refer back to Example CFNLT. We decomposed the vector space ${ℂ}^6$ into a direct sum of the subspaces $Z_1,Z_2,Z_3,Z_4$. The union of the basis vectors for these subspaces is a basis of ${ℂ}^6$, which we reordered prior to building a matrix representation of the linear transformation $T$. A principal reason for this reordering was to create invariant subspaces (though it was not obvious then).

Define

Recall from the proof of Theorem CFNLT or the computations in Example CFNLT that first elements of $X_1$ and $X_2$ are in the kernel of $T$, $\mathcal{K}\left(T\right)$, and each element of $X_1$ and $X_2$ is the output of $T$ when evaluated with the subsequent element of the set. This was by design, and it is this feature of these basis vectors that leads to the nearly diagonal matrix representation with Jordan blocks. However, we also recognize now that this property of these basis vectors allow us to conclude easily that $X_1$ and $X_2$ are invariant subspaces of ${ℂ}^6$ relative to $T$.

Furthermore, ${ℂ}^6=X_1\oplus X_2$ (Theorem DSFB). So the domain of $T$ is the direct sum of invariant subspaces and the resulting matrix representation has a block diagonal form. Hmmmmm. $⊠$

Subsection GEE: Generalized Eigenvectors and Eigenspaces

We now define a new type of invariant subspace and explore its key properties. This generalization of eigenvalues and eigenspaces will allow us to move from diagonal matrix representations of diagonalizable matrices to nearly diagonal matrix representations of arbitrary matrices. Here are the definitions.

Definition GEV
Generalized Eigenvector
Suppose that $T:V\mapsto V$ is a linear transformation. Suppose further that for $x\ne 0$, ${\left(T-\lambda I_V\right)}^k\left(x\right)=0$ for some $k>0$. Then $x$ is a generalized eigenvector of $T$ with eigenvalue $\lambda$. $△$

Definition GES
Generalized Eigenspace
Suppose that $T:V\mapsto V$ is a linear transformation. Define the generalized eigenspace of $T$ for $\lambda$ as

(This definition contains Notation GES.) $△$

So the generalized eigenspace is composed of generalized eigenvectors, plus the zero vector. As the name implies, the generalized eigenspace is a subspace of $V$. But more topically, it is an invariant subspace of $V$ relative to $T$.

Theorem GESIS
Generalized Eigenspace is an Invariant Subspace
Suppose that $T:V\mapsto V$ is a linear transformation. Then the generalized eigenspace ${\mathcal{G}}_T\left(\lambda \right)$ is an invariant subspace of $V$ relative to $T$. $\square$

Proof   First we establish that ${\mathcal{G}}_T\left(\lambda \right)$ is a subspace of $V$. First ${\left(T-\lambda I_V\right)}^1\left(0\right)=0$ by Theorem LTTZZ, so $0\in {\mathcal{G}}_T\left(\lambda \right)$.

Suppose that $x,y\in {\mathcal{G}}_T\left(\lambda \right)$. Then there are integers $k,\ell$ such that ${\left(T-\lambda I_V\right)}^k\left(x\right)=0$ and ${\left(T-\lambda I_V\right)}^{\ell }\left(y\right)=0$. Set $m=k+\ell$,

So $x+y\in {\mathcal{G}}_T\left(\lambda \right)$.

Suppose that $x\in {\mathcal{G}}_T\left(\lambda \right)$ and $\alpha \in ℂ$. Then there is an integer $k$ such that ${\left(T-\lambda I_V\right)}^k\left(x\right)=0$.

So $\alpha x\in {\mathcal{G}}_T\left(\lambda \right)$. By Theorem TSS, ${\mathcal{G}}_T\left(\lambda \right)$ is a subspace of $V$.

Now we show that ${\mathcal{G}}_T\left(\lambda \right)$ is invariant relative to $T$. Suppose that $x\in {\mathcal{G}}_T\left(\lambda \right)$. Then there is an integer $k$ such that ${\left(T-\lambda I_V\right)}^k\left(x\right)=0$. Recognize also that ${\left(T-\lambda I_V\right)}^k$ is a polynomial in $T$, and therefore commutes with $T$ (that is, $T\circ p\left(T\right)=p\left(T\right)\circ T$ for any polynomial $p\left(x\right)$). Now,

This qualifies $T\left(x\right)$ for membership in ${\mathcal{G}}_T\left(\lambda \right)$, so by Definition GES, ${\mathcal{G}}_T\left(\lambda \right)$ is invariant relative to $T$. $■$

Before we compute some generalized eigenspaces, we state and prove one theorem that will make it much easier to create a generalized eigenspace, since it will allow us to use tools we already know well, and will remove some the ambiguity of the clause “for some $k$” in the definition.

Theorem GEK
Generalized Eigenspace as a Kernel
Suppose that $T:V\mapsto V$ is a linear transformation, $dim\left(V\right)=n$, and $\lambda$ is an eigenvalue of $T$. Then ${\mathcal{G}}_T\left(\lambda \right)=\mathcal{K}\left({\left(T-\lambda I_V\right)}^n\right)$. $\square$

Proof   The conclusion of this theorem is a set equality, so we will apply Definition SE by establishing two set inclusions. First, suppose that $x\in {\mathcal{G}}_T\left(\lambda \right)$. Then there is an integer $k$ such that ${\left(T-\lambda I_V\right)}^k\left(x\right)=0$. This is equivalent to the statement that $x\in \mathcal{K}\left({\left(T-\lambda I_V\right)}^k\right)$. No matter what the value of $k$ is, Theorem KPLT gives

So, ${\mathcal{G}}_T\left(\lambda \right)\subseteq \mathcal{K}\left({\left(T-\lambda I_V\right)}^n\right)$. For the opposite inclusion, suppose $y\in \mathcal{K}\left({\left(T-\lambda I_V\right)}^n\right)$. Then ${\left(T-\lambda I_V\right)}^n\left(y\right)=0$, so $y\in {\mathcal{G}}_T\left(\lambda \right)$ and thus $\mathcal{K}\left({\left(T-\lambda I_V\right)}^n\right)\subseteq {\mathcal{G}}_T\left(\lambda \right)$. By Definition SE we have the desired equality of sets. $■$

Theorem GEK allows us to compute generalized eigenspaces as a single kernel (or null space of a matrix representation) with tools like Theorem KNSI and Theorem BNS. Also, we do not need to consider all possible powers $k$ and can simply consider the case where $k=n$. It is worth noting that the “regular” eigenspace is a subspace of the generalized eigenspace since

where the subset inclusion is a consequence of Theorem KPLT. Also, there is no such thing as a “generalized eigenvalue.” If $\lambda$ is not an eigenvalue of $T$, then the kernel of $T-\lambda I_V$ is trivial and therefore subsequent powers of $T-\lambda I_V$ also have trivial kernels (Theorem KPLT). So the generalized eigenspace of a scalar that is not already an eigenvalue would be trivial. Alright, we know enough now to compute some generalized eigenspaces. We will record some information about algebraic and geometric multiplicities of eigenvalues (Definition AME, Definition GME) as we go, since these observations will be of interest in light of some future theorems.

Example GE4
Generalized eigenspaces, dimension 4 domain
In Example TIS we presented two invariant subspaces of ${ℂ}^4$. There was some mystery about just how these were constructed, but we can now reveal that they are generalized eigenspaces. Example TIS featured $T:{ℂ}^4\mapsto {ℂ}^4$ defined by $T\left(x\right)=Ax$ with $A$ given by

A matrix representation of $T$ relative to the standard basis (Definition SUV) will equal $A$. So we can analyze $A$ with the techniques of Chapter E. Doing so, we find two eigenvalues, $\lambda =1,-2$, with multiplicities,

To apply Theorem GEK we subtract each eigenvalue from the diagonal entries of $A$, raise the result to the power $dim\left({ℂ}^4\right)=4$, and compute a basis for the null space.

In Example TIS we concluded that these two invariant subspaces formed a direct sum of ${ℂ}^4$, only at that time, they were called $X$ and $W$. Now we can write

This is no accident. Notice that the dimension of each of these invariant subspaces is equal to the algebraic multiplicity of the associated eigenvalue. Not an accident either. (See the upcoming Theorem GESD.) $⊠$

Example GE6
Generalized eigenspaces, dimension 6 domain
Define the linear transformation $S:{ℂ}^6\mapsto {ℂ}^6$ by $S\left(x\right)=Bx$ where

Then $B$ will be the matrix representation of $S$ relative to the standard basis (Definition SUV) and we can use the techniques of Chapter E applied to $B$ in order to find the eigenvalues of $S$.

To find the generalized eigenspaces of $S$ we need to subtract an eigenvalue from the diagonal elements of $B$, raise the result to the power $dim\left({ℂ}^6\right)=6$ and compute the null space. Here are the results for the two eigenvalues of $S$,

If we take the union of the two bases for these two invariant subspaces we obtain the set

You can check that this set is linearly independent (right now we have no guarantee this will happen). Once this is verified, we have a linearly independent set of size 6 inside a vector space of dimension 6, so by Theorem G, the set $C$ is a basis for ${ℂ}^6$. This is enough to apply Theorem DSFB and conclude that

This is no accident. Notice that the dimension of each of these invariant subspaces is equal to the algebraic multiplicity of the associated eigenvalue. Not an accident either. (See the upcoming Theorem GESD.) $⊠$

Subsection RLT: Restrictions of Linear Transformations

Generalized eigenspaces will prove to be an important type of invariant subspace. A second reason for our interest in invariant subspaces is they provide us with another method for creating new linear transformations from old ones.

Definition LTR
Linear Transformation Restriction
Suppose that $T:V\mapsto V$ is a linear transformation, and $U$ is an invariant subspace of $V$ relative to $T$. Define the restriction of $T$ to $U$ by

(This definition contains Notation LTR.) $△$

It might appear that this definition has not accomplished anything, as $T{|}_U$ would appear to take on exactly the same values as $T$. And this is true. However, $T{|}_U$ differs from $T$ in the choice of domain and codomain. We tend to give little attention to the domain and codomain of functions, while their defining rules get the spotlight. But the restriction of a linear transformation is all about the choice of domain and codomain. We are restricting the rule of the function to a smaller subspace. Notice the importance of only using this construction with an invariant subspace, since otherwise we cannot be assured that the outputs of the function are even contained in the codomain. Maybe this observation should be the key step in the proof of a theorem saying that $T{|}_U$ is also a linear transformation, but we won’t bother.

Example LTRGE
Linear transformation restriction on generalized eigenspace
In order to gain some expereince with restrictions of linear transformations, we construct one and then also construct a matrix representation for the restriction. Furthermore, we will use a generalized eigenspace as the invariant subspace for the construction of the restriction.

Consider the linear transformation $T:{ℂ}^5\mapsto {ℂ}^5$ defined by $T\left(x\right)=Ax$, where

One of the eigenvalues of $A$ is $\lambda =2$, with geometric multiplicity ${\gamma }_T\left(2\right)=1$, and algebraic multiplicity ${\alpha }_T\left(2\right)=3$. We get the generalized eigenspace in the usual manner,

By Theorem GESIS, we know $W$ is invariant relative to $T$, so we can employ Definition LTR to form the restriction, $T{|}_W:W\mapsto W$.

To better understand exactly what a restriction is (and isn’t), we’ll form a matrix representation of $T{|}_W$. This will also be a skill we will use in subsequent examples. For a basis of $W$ we will use $C=\left\{w_1,w_2,w_3\right\}$. Notice that $dim\left(W\right)=3$, so our matrix representation will be a square matrix of size 3. Applying Definition MR, we compute

So the matrix representation of $T{|}_W$ relative to $C$ is

The question arises: how do we use a $3\times 3$ matrix to compute with vectors from ${ℂ}^5$? To answer this question, consider the randomly chosen vector

First check that $w\in {\mathcal{G}}_T\left(2\right)$. There are two ways to do this, first verify that

meeting Definition GES (with $k=5$). Or, express $w$ as a linear combination of the basis $C$ for $W$, to wit, $w=4w_1-2w_2-w_3$. Now compute $T{|}_W\left(w\right)$ directly using Definition LTR,

It was necessary to verify that $w\in {\mathcal{G}}_T\left(2\right)$, and if we trust our work so far, then this output will also be an element of $W$, but it would be wise to check this anyway (using either of the methods we used for $w$). We’ll wait.

Now we will repeat this sample computation, but instead using the matrix representation of $T{|}_W$ relative to $C$.

which matches the previous computation. Notice how the “action” of $T{|}_W$ is accomplished by a $3\times 3$ matrix multiplying a column vector of size 3. If you would like more practice with these sorts of computations, mimic the above using the other eigenvalue of $T$, which is $\lambda =-2$. The generalized eigenspace has dimension 2, so the matrix representation of the restriction to the generalized eigenspace will be a $2\times 2$ matrix. $⊠$

Suppose that $T:V\mapsto V$ is a linear transformation and we can find a decomposition of $V$ as a direct sum, say $V=U_1\oplus U_2\oplus U_3\oplus \cdots \oplus U_m$ where each $U_i$ is an invariant subspace of $V$ relative to $T$. Then, for any $v\in V$ there is a unique decomposition $v=u_1+u_2+u_3+\cdots +u_m$ with $u_i\in U_i$, $1\le i\le m$ and furthermore

So in a very real sense, we obtain a decomposition of the linear transformation $T$ into the restrictions $T{|}_{U_i}$, $1\le i\le m$. If we wanted to be more careful, we could extend each restriction to a linear transformation defined on $V$ by setting the output of $T{|}_{U_i}$ to be the zero vector for inputs outside of $U_i$. Then $T$ would be exactly equal to the sum (Definition LTA) of these extended restrictions. However, the irony of extending our restrictions is more than we could handle right now.

Our real interest is in the matrix representation of a linear transformation when the domain decomposes as a direct sum of invariant subspaces. Consider forming a basis $B$ of $V$ as the union of bases $B_i$ from the individual $U_i$, i.e. $B={\cup }_{i=1}^m{B}_i$. Now form the matrix representation of $T$ relative to $B$. The result will be block diagonal, where each block is the matrix representation of a restriction $T{|}_{U_i}$ relative to a basis $B_i$, $M_{B_i,B_i}^{T{|}_{U_i}}$. Though we did not have the definitions to describe it then, this is exactly what was going on in the latter portion of the proof of Theorem CFNLT. Two examples should help to clarify these ideas.

Example ISMR4
Invariant subspaces, matrix representation, dimension 4 domain
Example TIS and Example GE4 describe a basis of ${ℂ}^4$ which is derived from bases for two invariant subspaces (both generalized eigenspaces). In this example we will construct a matrix representation of the linear transformation $T$ relative to this basis. Recycling the notation from Example TIS, we work with the basis,

Now we compute the matrix representation of $T$ relative to $B$, borrowing some computations from Example TIS,