Section JCF  Jordan Canonical Form

From A First Course in Linear Algebra
Version 1.07
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/

This Section is a Draft
We have seen in Section IS that generalized eigenspaces are invariant subspaces that in every instance have led to a direct sum decomposition of the domain of the associated linear transformation. This allows us to create a block diagonal matrix representation (Example ISMR4, Example ISMR6). We also know from Theorem RGEN that the restriction of a linear transformation to a generalized eigenspace is almost a nilpotent linear transformation. Of course, we understand nilpotent linear transformations very well from Section NLT and we have carefully determined a nice matrix representation for them.

So here is the game plan for the final push. Prove that the domain of a linear transformation always decomposes into a direct sum of generalized eigenspaces. We have unravelled Theorem RGEN at Theorem MRRGE so that we can formulate the matrix representations of the restrictions on the generalized eigenspaces using our storehouse of results about nilpotent linear transformations. Arrive at a matrix representation of any linear transformation that is block diagonal with each block being a Jordan block.

Subsection GESD: Generalized Eigenspace Decomposition

In Theorem UTMR we were able to show that any linear transformation from $V$ to $V$ has an upper triangular matrix representation (Theorem UTM). We will now show that we can improve on the basis yielding this representation by massaging the basis so that the matrix representation is also block diagonal. The subspaces associated with each block will be generalized eigenspaces, so the most general result will be a decomposition of the domain of a linear transformation into a direct sum of generalized eigenspaces.

Theorem GESD
Generalized Eigenspace Decomposition
Suppose that $T\left(V\right)V$ is a linear transformation with distinct eigenvalues ${\lambda }_1,{\lambda }_2,{\lambda }_3,\dots ,{\lambda }_m$. Then

Proof   Suppose that $dim\left(V\right)=n$ and the $n$ (not necessarily distinct) eigenvalues of $T$ are $scalarlist\rho n$. We begin with a basis of $V$ that yields an upper triangular matrix representation, as guaranteed by Theorem UTMR, $B=\left\{x_1,x_2,x_3,\dots ,x_n\right\}$. Since the matrix representation is upper triangular, and the eigenvalues of the linear transformation are the diagonal elements we can choose this basis so that there are then scalars $a_{ij}$, $1\le j\le n$, $1\le i\le j-1$ such that

We now define a new basis for $V$ which is just a slight variation in the basis $B$. Choose any $k$ and $\ell$ such that $1\le k<\ell \le n$ and ${\rho }_k\ne {\rho }_{\ell }$. Define the scalar $\alpha =a_{kl}∕\left({\rho }_{\ell }-{\rho }_k\right)$. The new basis is $C=\left\{y_1,y_2,y_3,\dots ,y_n\right\}$ where

We now compute the values of the linear transformation $T$ with inputs from $C$, noting carefully the changed scalars in the linear combinations of $C$ describing the outputs. These changes will translate to minor changes in the matrix representation built using the basis $C$. There are three cases to consider, depending on which column of the matrix representation we are examining. First, assume $j<\ell$. Then

That seems a bit pointless. The first $\ell -1$ columns of the matrix representations of $T$ relative to $B$ and $C$ are identical. OK, if that was too easy, here’s the main act. Assume $j=\ell$. Then

So how different are the matrix representations relative to $B$ and $C$ in column $\ell$? For $i>k$, the coefficient of $y_i$ is $a_{ij}$, as in the representation relative to $B$. It is a different story for $i\le k$, where the coefficients of $y_i$ may be very different. We are especially interested in the coefficient of $y_k$. In fact, this whole first part of this proof is about this particular entry of the matrix representation. The coefficient of $y_k$ is

If the definition of $\alpha$ was a mystery, then no more. In the matrix representation of $T$ relative to $C$, the entry in column $\ell$, row $k$ is a zero. Nice. The only price we pay is that other entries in column $\ell$, specifically rows $1$ through $k-1$, may also change in a way we can’t control.

One more case to consider. Assume $j>\ell$. Then

As before, we ask: how different are the matrix representations relative to $B$ and $C$ in column $j$? Only $y_k$ has a coefficent different from the corresponding coefficient when the basis is $B$. So in the matrix representations, the only entries to change are in row $k$, for columns $\ell +1$ through $n$.

What have we accomplished? With a change of basis, we can place a zero in a desired entry (row $k$, column $\ell$) of the matrix representation, leaving most of the entries untouched. The only entries to possibly change are above the new zero entry, or to the right of the new zero entry. S Suppose we repeat this procedure, starting by “zeroing out” the entry above the diagonal in the second column and first wow. Then we move right to the third column, and zero out the element just above the diagonal in the second row. Next we zero out the element in the third column and first row. Then tackle the fourth column, work upwards from the diagonal, zeroing out elements as we go. Entries above, and to the right will repeatedly change, but newly created zeros will never get wrecked, since they are below, or just to the left of the entry we are working on. Similarly the values on the diagonal do not change either. This entire argument can be retooled in the language of change-of-basis matrices and similarity transformations, and this is the approach taken by Noble in his Applied Linear Algebra. It is interesting to concoct the change-of-basis matrix between the matrices $B$ and $C$ and compute the inverse.

Perhaps you have noticed that we have to be just a bit more careful than the previous paragraph suggests. The definition of $\alpha$ has a denominator that cannot be zero, which restricts our maneuvers to zeroing out entries in row $k$ and column $\ell$ only when ${\rho }_k\ne {\rho }_{\ell }$. So we do not necessarily arrive at a diagonal matrix. More carefully we can write

where the $b_{ij}$ are our new coefficients after repeated changes, the $y_j$ are the new basis vectors, and the condition “$i:{\rho }_i={\rho }_j$” means that we only have terms in the sum involving vectors whose final coefficients are identical diagonal values (the eigenvalues). Now reorder the basis vectors carefully. Group together vectors that have equal diagonal entries in the matrix representation, but within each group preserve the order of the precursor basis. This grouping will create a block diagonal structure for the matrix representation, while otherwise preserving the order of the basis will retain the upper triangular form of the representation. So we can arrive at a basis that yields a matrix representation that is upper triangular and block diagonal, with the diagonal entries of each block all equal to a common eigenvalue of the linear transformation.

More carefully, employing the distinct eigenvalues of $T$, ${\lambda }_i$, $1\le i\le m$, we can assert there is a set of basis vectors for $V$, $u_{ij}$, $1\le i\le m$, $1\le j\le {\alpha }_T\left({\lambda }_i\right)$, such that

So the subspace $U_i=〈\left\{\left.u_{ij}\right|1\le j\le {\alpha }_T\left({\lambda }_i\right)\right\}〉$, $1\le i\le m$ is an invariant subspace of $V$ relative to $T$ and the restriction $T{|}_{U_i}$ has an upper triangular matrix representation relative to the basis $\left\{\left.u_{ij}\right|1\le j\le {\alpha }_T\left({\lambda }_i\right)\right\}$ where the diagonal entries are all equal to ${\lambda }_i$. Notice too that with this definition,

Whew. This is a good place to take a break, grab a cup of coffee, use the toilet, or go for a short stroll, before we show that $U_i$ is a subspace of the generalized eigenspace ${\mathcal{G}}_T\left({\lambda }_i\right)$. This will follow if we can prove that each of the basis vectors for $U_i$ is a generalized eigenvector of $T$ for ${\lambda }_i$ (Definition GEV). We need some power of $T-{\lambda }_i{I}_V$ that takes $u_{ij}$ to the zero vector. We prove by induction on $j$ (Technique I) the claim that ${\left(T-{\lambda }_i{I}_V\right)}^j\left(u_{ij}\right)=0$. For $j=1$ we have,

For the induction step, assume that if $k<j$, then ${\left(T-{\lambda }_i{I}_V\right)}^k$ takes $u_{ik}$ to the zero vector. Then

This completes the induction step. Since every vector of the spaning set for $U_i$ is an element of the subspace ${\mathcal{G}}_T\left({\lambda }_i\right)$, Property AC and Property SC allow us to conclude that $U_i\subseteq {\mathcal{G}}_T\left({\lambda }_i\right)$. Then by Definition S, $U_i$ is a subspace of ${\mathcal{G}}_T\left({\lambda }_i\right)$. Notice that this inductive proof could be interpreted to say that every element of $U_i$ is a generalized eigenvector of $T$ for ${\lambda }_i$, and the algebraic multiplicity of ${\lambda }_i$ is a sufficiently high power to demonstrate this via the definition for each vector.

We are now prepared for our final argument in this long proof. We wish to establish that the dimension of the subspace ${\mathcal{G}}_T\left({\lambda }_i\right)$ is the algebraic multiplicity of ${\lambda }_i$. This will be enough to show that $U_i$ and ${\mathcal{G}}_T\left({\lambda }_i\right)$ are equal, and will finally provide the desired direct sum decomposition.

We will prove by induction (Technique I) the following claim. Suppose that $T:V\mapsto V$ is a linear transformation and $B$ is a basis for $V$ that provides an upper triangular matrix representation of $T$. The number of times any eigenvalue $\lambda$ occurs on the diagonal of the representation is greater than or equal to the dimension of the generalized eigenspace ${\mathcal{G}}_T\left(\lambda \right)$.

We will use the symbol $m$ for the dimension of $V$ so as to avoid confusion with our notation for the nullity. So $dimV=m$ and our proof will proced by induction on $m$. Use the notation ${\#}_T\left(\lambda \right)$ to count the number of times $\lambda$ occurs on the diagonal of a matrix representation of $T$. We want to show that

For the base case, $dimV=1$. Every matrix representation of $T$ is an upper triangular matrix with the lone eigenvalue of $T$, $\lambda$, as the diagonal entry. So ${\#}_T\left(\lambda \right)=1$. The generalized eigenspace of $\lambda$ is not trivial (since by Theorem GEK it equals the regular eigenspace), and is a subspace of $V$. With Theorem PSSD we see that $dim\left({\mathcal{G}}_T\left(\lambda \right)\right)=1$.

Now for the induction step, assume the claim is true for any linear transformation defined on a vector space with dimension $m-1$ or less. Suppose that $B=\left\{v_1,v_2,v_3,\dots ,v_m\right\}$ is a basis for $V$ that yields a diagonal matrix representation for $T$ with diagonal entries ${\lambda }_1,{\lambda }_2,{\lambda }_3,\dots ,{\lambda }_m$. Then $U=〈\left\{v_1,v_2,v_3,\dots ,v_{m-1}\right\}〉$ is a subspace of $V$ that is invariant relative to $T$. The restriction $T{|}_U:U\mapsto U$ is then a linear transformation defined on $U$, a vector space of dimension $m-1$. A matrix representation of $T{|}_U$ relative to the basis $C=\left\{v_1,v_2,v_3,\dots ,v_{m-1}\right\}$ will be an upper triangular matrix with diagonal entries ${\lambda }_1,{\lambda }_2,{\lambda }_3,\dots ,{\lambda }_{m-1}$. We can therefore apply the induction hypothesis to $T{|}_U$ and its representation relative to $C$.

Suppose that $\lambda$ is any eigenvalue of $T$. Then suppose that $v\in \mathcal{K}\left({\left(T-{\lambda }_V\right)}^m\right)$. As an element of $V$, we can write $v$ as a linear combination of the basis elements of $B$, or more compactly, there is a vector $u\in U$ and a scalar $\alpha$ such that $v=u+\alpha v_m$. Then,

The final expression in this string of equalities is an element of $U$ since $U$ is invariant relative to both $T$ and $I_V$. The expression at the beginning is a scalar multiple of $v_m$, and as such cannot be a nonzero element of $U$ without violating the linear independence of $B$. So

The vector $v_m$ is nonzero since $B$ is linearly independent, so Theorem SMEZV tells us that $\alpha {\left({\lambda }_m-\lambda \right)}^m=0$. From the properties of scalar multiplication, we are confronted with two possibilities.

Our first case is that $\lambda \ne {\lambda }_m$. Notice then that $\lambda$ occurs the same number of times along the diagonal in the representations of $T{|}_U$ and $T$. Now $\alpha =0$ and $v=u+0v_m=u$. Since $v$ was chosen as an arbitrary element of $\mathcal{K}\left({\left(T-\lambda I_V\right)}^m\right)$, Definition SSET says that $\mathcal{K}\left({\left(T-\lambda I_V\right)}^m\right)\subseteq U$. It is always the case that $\mathcal{K}\left({\left(T{|}_U-\lambda I_U\right)}^m\right)\subseteq \mathcal{K}\left({\left(T-\lambda I_V\right)}^m\right)$. However, we can also see that in this case, the opposite set inclusion is true as well. By Definition SE we have $\mathcal{K}\left({\left(T{|}_U-\lambda I_U\right)}^m\right)=\mathcal{K}\left({\left(T-\lambda I_V\right)}^m\right)$. Then

The second case is that $\lambda ={\lambda }_m$. Notice then that $\lambda$ occurs one more time along the diagonal in the representation of $T$ compared to the representation of $T{|}_U$. Then

So $u\in \mathcal{K}\left(T{|}_U-\lambda I_U\right)$. The vector $v$ is an arbitrary member of $\mathcal{K}\left({\left(T-\lambda I_V\right)}^m\right)$ and is also equal to an element of $\mathcal{K}\left(T{|}_U-\lambda I_U\right)$ ($u$) plus a scalar multiple of the vector $v_m$. This observation yields

Now count eigenvalues on the diagonal,

In Theorem UTMR we constructed an upper triangular matrix represntation of $T$ where each eigenvalue occurred ${\alpha }_T\left(\lambda \right)$ times on the diagonal. So

Thus, $dim\left({\mathcal{G}}_T\left({\lambda }_i\right)\right)={\alpha }_T\left({\lambda }_i\right)$ and by Theorem EDYES, $U_i={\mathcal{G}}_T\left({\lambda }_i\right)$ and we can write

Besides a nice decomposition into invariant subspaces, this proof has a bonus for us.

Theorem DGES
Dimension of Generalized Eigenspaces
Suppose $T:V\mapsto V$ is a linear transformation with eigenvalue $\lambda$. Then the dimension of the generalized eigenspace for $\lambda$ is the algebraic multiplicity of $\lambda$, $dim\left({\mathcal{G}}_T\left({\lambda }_i\right)\right)={\alpha }_T\left({\lambda }_i\right)$. $\square$

Proof   At the very end of the proof of Theorem GESD we obtain the inequalities

which establishes the desired equality. $■$

Subsection JCF: Jordan Canonical Form

Now we are in a position to define what we (and others) regard as an especially nice matrix representation. The word “canonical” has at its root, the word “canon,” which has various meanings. One is the set of laws established by a church council. Another is a set of writings that are authentic, important or representative. Here we take to to mean the accepted, or best, representative among a variety of choices. Every linear transformation admits a variety of representations, and will declare one as the best. Hopefully you will agree.

Definition JCF
Jordan Canonical Form
A square matrix is in Jordan canonical form if it meets the following requirements:

1. The matrix is block diagonal.
2. Each block is a Jordan block.
3. If $\rho <\lambda$ then the block $J_k\left(\rho \right)$ occupies rows with indices greater than the indices of the rows occupied by $J_{\ell }\left(\lambda \right)$.
4. If $\rho =\lambda$ and $\ell <k$, then the block $J_{\ell }\left(\lambda \right)$ occupies rows with indices greater than the indices of the rows occupied by $J_k\left(\lambda \right)$.

Theorem JCFLT
Jordan Canonical Form for a Linear Transformation
Suppose $T:V\mapsto V$ is a linear transformation. Then there is a basis $B$ for $V$ such that the matrix representation of $T$ with the following properties:

1. The matrix representation is in Jordan canonical form.
2. If $J_k\left(\lambda \right)$ is one of the Jordan blocks, then $\lambda$ is an eigenvalue of $T$.
3. For a fixed value of $\lambda$, the largest block of the form $J_k\left(\lambda \right)$ has size equal to the index of $\lambda$, ${\iota }_T\left(\lambda \right)$.
4. For a fixed value of $\lambda$, the number of blocks of the form $J_k\left(\lambda \right)$ is the geometric multiplicity of $\lambda$, ${\gamma }_T\left(\lambda \right)$.
5. For a fixed value of $\lambda$, the number of rows occupied by blocks of the form $J_k\left(\lambda \right)$ is the algebraic multiplicity of $\lambda$, ${\alpha }_T\left(\lambda \right)$.

Proof   This theorem is really just the consequence of applying to $T$, consecutively Theorem GESD, Theorem MRRGE and Theorem CFNLT.

Theorem GESD gives us a decomposition of $V$ into generalized eigenspaces, one for each distinct eigenvalue. Since these generalized eigenspaces ar invariant relative to $T$, this provides a block diagonal matrix representation where each block is the matrix representation of the restriction of $T$ to the generalized eigenspace.

Restricting $T$ to a generalized eigenspace results in a “nearly nilpotent” linear transformation, as stated more precisely in Theorem RGEN. We unravel Theorem RGEN in the proof of Theorem MRRGE so that we can apply Theorem CFNLT about representations of nilpotent linear transformations.

We know the dimension of a generalized eigenspace is the algebraic multiplicity of the eigenvalue (Theorem DGES), so the blocks associated with the generalized eigenspaces are square with a size equal to the algebraic multiplicity. In refining the basis for this block, and producing Jordan blocks the results of Theorem CFNLT apply. The total number of blocks will be the nullity of $T{|}_{{\mathcal{G}}_T\left(\lambda \right)}-\lambda I_{{\mathcal{G}}_T\left(\lambda \right)}$, which is the geometric multiplicity of $\lambda$ as an eigenvalue of $T$ (Definition GME). The largest of the Jordan blocks will have size equal to the index of the nilpotent linear transformation $T{|}_{{\mathcal{G}}_T\left(\lambda \right)}-\lambda I_{{\mathcal{G}}_T\left(\lambda \right)}$, which is exactly the definition of the index of the eigenvalue $\lambda$ (Definition IE). $■$

Before we do some examples of this result, notice how close Jordan canonical form is to a diagonal matrix. Or, equivalently, notice how close we have come to diagonalizing a matrix (Definition DZM). We have a matrix representation which has diagonal entries that are the eigenvalues of a matrix. Each occurs on the diagonal as many times as the algebraic multiplicity. However, when the geometric multiplicty is strictly less than the algebraic multiplicity, we have some entries in the representation just above the diagonal (the “superdiagonal”). Furthermore, we have some idea how often this happens if we know the geometric multiplicity and the index of the eigenvalue.

We now recognize just how simple a diagonalizable linear transformation really is. For each eigenvalue, the generalized eigenspace is just the regular eigenspace, and it decomposes into a direct sum of one-dimensional subspaces, each spanned by a different eigenvector chosen from a basis of eigenvectors for the eigenspace.

Some authors create matrix representations of nilpotent linear transformations where the Jordan block has the ones just below the diagonal (the “subdiagonal”). No matter, it is really the same, just different. We have also defined Jordan canonical form to place blocks for the larger eigenvalues earlier, and for blocks with the same eigenvalue, we place the bigger ones earlier. This is fairly standard, but there is no reason we couldn’t order the blocks differently. It’d be the same, just different. The reason for choosing some ordering is to be assured that there is just one canonical matrix representation for each linear transformation.

Example JCF10
Jordan canonical form, size 10
Suppose that $T:{ℂ}^{10}\mapsto {ℂ}^{10}$ is the linear transformation defined by $T\left(x\right)=Ax$ where

We’ll find a basis for ${ℂ}^{10}$ that will yield a matrix representation of $T$ in Jordan canonical form. First we find the eigenvalues, and their multiplicities, with the techniques of Chapter E.

For each eigenvalue, we can compute a generalized eigenspace. By Theorem GESD we know that ${ℂ}^{10}$ will decompose into a direct sum of these eigenspaces, and we can restrict $T$ to each part of this decomposition. At this stage we know that the Jordan canonical form will be block diagonal with blocks of size $2$, $3$ and $5$, since the dimensions of the generalized eigenspaces are equal to the algebraic multiplicities of the eigenvalues (Theorem DGES). The geometric multiplicities tell us how many Jordan blocks occupy each of the three larger blocks, but we will discuss this as we analyze each eigenvalue. We do not yet know the index of each eigenalue (though we can easily infer it for $\lambda =2$) and even if we did have this information, it only determines the size of the largest Jordan block (per eigenvalue). We will press ahead, considering each eigenvalue one at a time.

The eigenvalue $\lambda =2$ has “full” geometric multiplicity, and is not an impediment to diagonalizing $T$. We will treat it in full generality anyway. First we compute the generalized eigenspace. Since Theorem GEK says that ${\mathcal{G}}_T\left(2\right)=\mathcal{K}\left({\left(T-2I_{{ℂ}^{10}}\right)}^{10}\right)$ we can compute this generalized eigenspace as a null space derived from the matrix $A$,

The restriction of $T$ to ${\mathcal{G}}_T\left(2\right)$ relative to the two basis vectors above has a matrix representation that is a $2\times 2$ diagonal matrix with the eigenvalue $\lambda =2$ as the diagonal entries. So these two vectors will be the first two vectors in our basis for ${ℂ}^{10}$,

Notice that it was not strictly necessary to compute the 10-th power of $A-2I_{10}$. With ${\alpha }_T\left(2\right)={\gamma }_T\left(2\right)$ the null space of the matrix $A-2I_{10}$ contains all of the generalized eigenvectors of $T$ for the eigenvalue $\lambda =2$. But there was no harm in computing the 10-th power either. This discussion is equivalent to the observation that the linear transformation $T{|}_{{\mathcal{G}}_T\left(2\right)}:{\mathcal{G}}_T\left(2\right)\mapsto {\mathcal{G}}_T\left(2\right)$ is nilpotent of index $1$. In other words, ${\iota }_T\left(2\right)=1$.

The eigenvalue $\lambda =0$ will not be quite as simple, since the geometric multiplicity is strictly less than the geometric multiplicity. As before, we first compute the generalized eigenspace. Since Theorem GEK says that ${\mathcal{G}}_T\left(0\right)=\mathcal{K}\left({\left(T-0I_{{ℂ}^{10}}\right)}^{10}\right)$ we can compute this generalized eigenspace as a null space derived from the matrix $A$,

So $dim\left({\mathcal{G}}_T\left(0\right)\right)=3={\alpha }_T\left(0\right)$, as expected. We will use these three basis vectors for the generalized eigenspace to construct a matrix representation of $T{|}_{{\mathcal{G}}_T\left(0\right)}$, where $F$ is being defined implicitly as the basis of ${\mathcal{G}}_T\left(0\right)$. We construct this representation as usual, applying Definition MR,