Section PD  Properties of Dimension

From A First Course in Linear Algebra
Version 0.94
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/

Once the dimension of a vector space is known, then the determination of whether or not a set of vectors is linearly independent, or if it spans the vector space, can often be much easier. In this section we will state a workhorse theorem and then apply it to the column space and row space of a matrix. It will also help us describe a super-basis for ${ℂ}^m$.

Subsection GT: Goldilocks’ Theorem

We begin with a useful theorem that we will need later, and in the proof of the main theorem in this subsection. This theorem says that we can extend linearly independent sets, one vector at a time, by adding vectors from outside the span of the linearly independent set, all the while preserving the linear independence of the set.

Theorem ELIS
Extending Linearly Independent Sets
Suppose $V$ is vector space and $S$ is a linearly independent set of vectors from $V$. Suppose $w$ is a vector such that $w\notin 〈S〉$. Then the set $S^{\prime }=S\cup \left\{w\right\}$ is linearly independent. $\square$

Proof   Suppose $S=\left\{v_1,v_2,v_3,\dots ,v_m\right\}$ and begin with a relation of linear dependence on $S^{\prime }$,

There are two cases to consider. First suppose that $a_{m+1}=0$. Then the relation of linear dependence on $S^{\prime }$ becomes

and by the linear independence of the set $S$, we conclude that $a_1=a_2=a_3=\cdots =a_m=0$. So all of the scalars in the relation of linear dependence on $S^{\prime }$ are zero.

In the second case, suppose that $a_{m+1}\ne 0$. Then the relation of linear dependence on $S^{\prime }$ becomes

This equation expresses $w$ as a linear combination of the vectors in $S$, contrary to the assumption that $w\notin 〈S〉$, so this case leads to a contradiction.

The first case yielded only a trivial relation of linear dependence on $S^{\prime }$ and the second case led to a contradiction. So $S^{\prime }$ is a linearly independent set since any relation of linear dependence is trivial. $■$

In the story Goldilocks and the Three Bears, the young girl Goldilocks visits the empty house of the three bears while out walking in the woods. One bowl of porridge is too hot, the other too cold, the third is just right. One chair is too hard, one too soft, the third is just right. So it is with sets of vectors — some are too big (linearly dependent), some are too small (they don’t span), and some are just right (bases). Here’s Goldilocks’ Theorem.

Theorem G
Goldilocks
Suppose that $V$ is a vector space of dimension $t$. Let $S=\left\{v_1,v_2,v_3,\dots ,v_m\right\}$ be a set of vectors from $V$. Then

1. If $m>t$, then $S$ is linearly dependent.
2. If $m<t$, then $S$ does not span $V$.
3. If $m=t$ and $S$ is linearly independent, then $S$ spans $V$.
4. If $m=t$ and $S$ spans $V$, then $S$ is linearly independent.

Proof   Let $B$ be a basis of $V$. Since $dim\left(V\right)=t$, Definition B and Theorem BIS imply that $B$ is a linearly independent set of $t$ vectors that spans $V$.

1. Suppose to the contrary that $S$ is linearly independent. Then $B$ is a smaller set of vectors that spans $V$. This contradicts Theorem SSLD.
2. Suppose to the contrary that $S$ does span $V$. Then $B$ is a larger set of vectors that is linearly independent. This contradicts Theorem SSLD.
3. Suppose to the contrary that $S$ does not span $V$. Then we can choose a vector $w$ such that $w\in V$ and $w\notin 〈S〉$. By Theorem ELIS, the set $S^{\prime }=S\cup \left\{w\right\}$ is again linearly independent. Then $S^{\prime }$ is a set of $m+1=t+1$ vectors that are linearly independent, while $B$ is a set of $t$ vectors that span $V$. This contradicts Theorem SSLD.
4. Suppose to the contrary that $S$ is linearly dependent. Then by Theorem DLDS (which can be upgraded, with no changes in the proof, to the setting of a general vector space), there is a vector in $S$, say $v_k$ that is equal to a linear combination of the other vectors in $S$. Let $S^{\prime }=S\setminus \left\{v_k\right\}$, the set of “other” vectors in $S$. Then it is easy to show that $V=〈S〉=〈S^{\prime }〉$. So $S^{\prime }$ is a set of $m-1=t-1$ vectors that spans $V$, while $B$ is a set of $t$ linearly independent vectors in $V$. This contradicts Theorem SSLD.

There is a tension in the construction of basis. Make a set too big and you will end up with relations of linear dependence among the vectors. Make a set too small and you will not have enough raw material to span the entire vector space. Make a set just the right size (the dimension) and you only need to have linear independence or spanning, and you get the other property for free. These roughly-stated ideas are made precise by Theorem G.

The structure and proof of this theorem also deserve comment. The hypotheses seem innocuous. We presume we know the dimension of the vector space in hand, then we mostly just look at the size of the set $S$. From this we get big conclusions about spanning and linear independence. Each of the four proofs relies on ultimately contradicting Theorem SSLD, so in a way we could think of this entire theorem as a corollary of Theorem SSLD. (See Technique LC.) The proofs of the third and fourth parts parallel each other in style (add $w$, toss $v_k$) and then turn on Theorem ELIS before contradicting Theorem SSLD.

Theorem G is useful in both concrete examples and as a tool in other proofs. We will use it often to bypass verifying linear independence or spanning.

Example BPR
Bases for $P_n$, reprised
In Example BP we claimed that

were both bases for $P_n$ (Example VSP). Suppose we had first verified that $B$ was a basis, so we would then know that $dim\left(P_n\right)=n+1$. The size of $C$ is $n+1$, the right size to be a basis. We could then verify that $C$ is linearly independent. We would not have to make any special efforts to prove that $C$ spans $P_n$, since Theorem G would allow us to conclude this property of $C$ directly. Then we would be able to say that $C$ is a basis of $P_n$ also. $⊠$

Example BDM22
Basis by dimension in $M_{22}$
In Example DSM22 we showed that

is a basis for the subspace $Z$ of $M_{22}$ (Example VSM) given by

This tells us that $dim\left(Z\right)=2$. In this example we will find another basis. We can construct two new matrices in $Z$ by forming linear combinations of the matrices in $B$.

Then the set

has the right size to be a basis of $Z$. Let’s see if it is a linearly independent set. The relation of linear dependence

leads to the homogeneous system of equations whose coefficient matrix

row-reduces to

So with $a_1=a_2=0$ as the only solution, the set is linearly independent. Now we can apply Theorem G to see that $C$ also spans $Z$ and therefore is a second basis for $Z$. $⊠$

Example SVP4
Sets of vectors in $P_4$
In Example BSP4 we showed that

is a basis for $W=\left\{\left.p\left(x\right)\right|p\in P_4,p\left(2\right)=0\right\}$. So $dim\left(W\right)=4$.

The set

is a subset of $W$ (check this) and it happens to be linearly independent (check this, too). However, by Theorem G it cannot span $W$.

The set

is another subset of $W$ (check this) and Theorem G tells us that it must be linearly dependent.

The set

is a third subset of $W$ (check this) and is linearly independent (check this). Since it has the right size to be a basis, and is linearly independent, Theorem G tells us that it also spans $W$, and therefore is a basis of $W$. $⊠$

Theorem ELIS allows us to extend a linearly independent set by adding an additional vector and preserving the linear independence of the set. This can be a useful technique in a proof, since properly employed, it saves us the trouble of checking linear independence. Rather than just adding one vector, we sometimes want to add several all at once. This is the main theme of the next theorem.

Theorem EB
Extending a Basis
Suppose that $W$ is a subspace of the vector space $V$. Then there exists a subspace $X$ of $V$ such that

1. For every $v\in V$ there exists vectors $w\in W$, $x\in X$ such that $v=w+x$.
2. $W\cap X=\left\{0\right\}$.

Proof   If $W=V$, then take $X=\left\{0\right\}$. Otherwise, suppose that $B=\left\{w_1,w_2,w_3,\dots ,w_m\right\}$ is a basis of $W$. Then $B$ is linearly independent and since $W⊊V$, Theorem ELIS gives the existence of a vector $x_1\notin W$ such that $B_1=\left\{w_1,w_2,w_3,\dots ,w_m,x_1\right\}$ is linearly independent. Define the subspace $W_1=〈B_1〉$, so $B_1$ is a basis of $W_1$. We can repeat this application of Theorem ELIS a second time, finding a vector $x_2$, leading to a basis $B_2$ for a new subspace $W_2$.

How many times can we extend the basis $B$? If $dim\left(V\right)=n$, then $B_{n-m}$ will be a linearly independent set of size $n$ in $V$. By Theorem G, $B_{n-m}$ is then a basis for $V$. Because $B_{n-m}$ is a spanning set for both $W_{n-m}$ and $V$, we see that $W_{n-m}=V$. At this point we will be unable to find $x_{n-m+1}\notin W_{n-m}$, and so cannot apply Theorem ELIS again. That’s OK, we have what we need.

Define $X=〈\left\{x_1,x_2,x_3,\dots ,x_{n-m}\right\}〉$. We need to establish two properties for $X$. First, suppose that $v$ is any vector in $V$, so $v\in V=W_{n-m}=〈B_{n-m}〉$. Thus there are scalars $a_1,a_2,a_3,\dots ,a_m$ and $b_1,b_2,b_3,\dots ,b_{m-n}$ so that

where we have implicitly defined the desired vectors $w$ and $x$. It should be clear that $w\in W$ and $x\in X$

Now for the second property, which is a set equality (Definition SE). Certainly $0\in W$ and $0\in X$ by Property Z, and thus $0\in W\cap X$ (Definition SI). Conversely, suppose that $z\in W\cap X$. Since $z\in W$, there exist scalars $a_1,a_2,a_3,\dots ,a_m$ such that

Also, since $z\in X$, there exist scalars $b_1,b_2,b_3,\dots ,b_{n-m}$ such that

Putting it all together,

This is a relation of linear dependence (Definition RLD) on the linearly independent set $B_{n-m}$ (Definition LI). Thus we can conclude that the scalars involved are all zero. Specifically, we just need $a_i=0$, $1\le i\le m$, so that

to establish that $W\cap X\subseteq \left\{0\right\}$. By Definition SE we have the second property. $■$

The relationships between $W$, $X$ and $V$ in the conclusion of Theorem EB are common enough that we form a definition to save us the trouble of repeating the whole conclusion repeatedly.

Definition DS
Direct Sum
Suppose that $V$ is a vector space with two subspaces $U$ and $W$ such that

1. For every $v\in V$ there exists vectors $u\in U$, $w\in W$ such that $v=u+w$
2. $U\cap W=\left\{0\right\}$

Then $V$ is the direct sum of $U$ and $W$ and we write $V=U\oplus W$.

(This definition contains Notation DS.) $△$

The second condition of this definition is often replaced by the equivalent condition that the vectors $u,w$ from the first condition must be unique. This formulation is sometimes easier to use when the notion of a direct sum is extended from a pair of subspaces to several. Some of the more advanced ideas in linear algebra are closely related to decomposing (Technique DC) vector spaces into direct sums of subspaces.

A simple consequence of Theorem G is the observation that proper subspaces have strictly smaller dimensions. Hopefully this may seem intuitively obvious, but it still requires proof, and we will cite this result later.

Theorem PSSD
Proper Subspaces have Smaller Dimension
Suppose that $U$ and $V$ are subspaces of the vector space $W$, such that $U⊊V$. Then $dim\left(U\right)<dim\left(V\right)$. $\square$

Proof   Suppose that $dim\left(U\right)=m$ and $dim\left(V\right)=t$. Then $U$ has a basis $B$ of size $m$. If $m>t$, then by Theorem G, $B$ is linearly dependent, which is a contradiction. If $m=t$, then by Theorem G, $B$ spans $V$. Then $U=〈B〉=V$, also a contradiction. All that remains is that $m<t$, which is the desired conclusion. $■$

The final theorem of this subsection is an extremely powerful tool for establishing the equality of two sets that are subspaces. Notice that the hypotheses include the equality of two integers (dimensions) while the conclusion is the equality of two sets (subspaces). It is the extra “structure” of a vector space and its dimension that makes possible this huge leap from an integer equality to a set equality.

Theorem EDYES
Equal Dimensions Yields Equal Subspaces
Suppose that $U$ and $V$ are subspaces of the vector space $W$, such that $U\subseteq V$ and $dim\left(U\right)=dim\left(V\right)$. Then $U=V$. $\square$

Proof   We give a proof by contradiction (Technique CD). Suppose to the contrary that $U\ne V$. Since $U\subseteq V$, there must be a vector $v$ such that $v\in V$ and $v\notin U$. Let $B=\left\{u_1,u_2,u_3,\dots ,u_t\right\}$ be a basis for $U$. Then, by Theorem ELIS, the set $C=B\cup \left\{v\right\}=\left\{u_1,u_2,u_3,\dots ,u_t,v\right\}$ is a linearly independent set of $t+1$ vectors in $V$. However, by hypothesis, $V$ has the same dimension as $U$ (namely $t$) and therefore Theorem G says that $C$ is too big to be linearly independent. This contradiction shows that $U=V$. $■$

Subsection RT: Ranks and Transposes

We now prove one of the most surprising theorems about matrices. Notice the paucity of hypotheses compared to the precision of the conclusion.

Theorem RMRT
Rank of a Matrix is the Rank of the Transpose
Suppose $A$ is an $m\times n$ matrix. Then $r\left(A\right)=r\left(A^t\right)$. $\square$

Proof   Suppose we row-reduce $A$ to the matrix $B$ in reduced row-echelon form, and $B$ has $r$ non-zero rows. The quantity $r$ tells us three things about $B$: the number of leading 1’s, the number of non-zero rows and the number of pivot columns. For this proof we will be interested in the latter two.

Theorem BRS and Theorem BCS each has a conclusion that provides a basis, for the row space and the column space, respectively. In each case, these bases contain $r$ vectors. This observation makes the following go.

This says that the row space and the column space of a matrix have the same dimension, which should be very surprising. It does not say that column space and the row space are identical. Indeed, if the matrix is not square, then the sizes (number of slots) of the vectors in each space are different, so the sets are not even comparable.

It is not hard to construct by yourself examples of matrices that illustrate Theorem RMRT, since it applies equally well to any matrix. Grab a matrix, row-reduce it, count the nonzero rows or the leading 1’s. That’s the rank. Transpose the matrix, row-reduce that, count the nonzero rows or the leading 1’s. That’s the rank of the transpose. The theorem says the two will be equal. Here’s an example anyway.

Example RRTI
Rank, rank of transpose, Archetype I
Archetype I has a $4\times 7$ coefficient matrix which row-reduces to

so the rank is $3$. Row-reducing the transpose yields

demonstrating that the rank of the transpose is also $3$. $⊠$

Subsection DFS: Dimension of Four Subspaces

That the rank of a matrix equals the rank of its transpose is a fundamental and surprising result. However, applying Theorem FS we can easily determine the dimension of all four fundamental subspaces associated with a matrix.

Theorem DFS
Dimensions of Four Subspaces
Suppose that $A$ is an $m\times n$ matrix, and $B$ is a row-equivalent matrix in reduced row-echelon form with $r$ nonzero rows. Then

1. $dim\left(\mathcal{N}\left(A\right)\right)=n-r$
2. $dim\left(\mathcal{C}\left(A\right)\right)=r$
3. $dim\left(\mathcal{ℛ}\left(A\right)\right)=r$
4. $dim\left(\mathcal{ℒ}\left(A\right)\right)=m-r$

Proof   If $A$ row-reduces to a matrix in reduced row-echelon form with $r$ nonzero rows, then the matrix $C$ of extended echelon form (Definition EEF) will be an $r\times n$ matrix in reduced row-echelon form with no zero rows and $r$ pivot columns (Theorem PEEF). Similarly, the matrix $L$ of extended echelon form (Definition EEF) will be an $m-r\times m$ matrix in reduced row-echelon form with no zero rows and $m-r$ pivot columns (Theorem PEEF).

There are many different ways to state and prove this result, and indeed, the equality of the dimensions of the column space and row space is just a slight expansion of Theorem RMRT. However, we have restricted our techniques to applying Theorem FS and then determining dimensions with bases provided by Theorem BNS and Theorem BRS. This provides an appealing symmetry to the results and the proof.

Subsection READ: Reading Questions

1. Why does Theorem G have the title it does?
2. What is so surprising about Theorem RMRT?
3. Row-reduce the matrix $A$ to reduced row-echelon form. Without any further computations, compute the dimensions of the four subspaces, $\mathcal{N}\left(A\right)$, $\mathcal{C}\left(A\right)$, $\mathcal{ℛ}\left(A\right)$ and $\mathcal{ℒ}\left(A\right)$.

   $$A=\left[\begin{array}{ccccc}\hfill 1\hfill & \hfill -1\hfill & \hfill 2\hfill & \hfill 8\hfill & \hfill 5\hfill \\ \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 4\hfill & \hfill -1\hfill \\ \hfill 0\hfill & \hfill 2\hfill & \hfill -3\hfill & \hfill -8\hfill & \hfill -6\hfill \\ \hfill 2\hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill 8\hfill & \hfill 4\hfill \end{array}\right]$$

Subsection EXC: Exercises

C10 Example SVP4 leaves several details for the reader to check. Verify these five claims.  
Contributed by Robert Beezer

T05 Trivially, if $U$ and $V$ are two subspaces of $W$, then $dim\left(U\right)=dim\left(V\right)$. Combine this fact, Theorem PSSD, and Theorem EDYES all into one grand combined theorem. You might look to Theorem PIP stylistic inspiration. (Notice this problem does not ask you to prove anything. It just asks you to roll up three theorems into one compact, logically equivalent statement.)  
Contributed by Robert Beezer

T10 Prove the following theorem, which could be viewed as a reformulation of parts (3) and (4) of Theorem G, or more appropriately as a corollary of Theorem G (Technique LC).

Suppose $V$ is a vector space and $S$ is a subset of $V$ such that the number of vectors in $S$ equals the dimendion of $V$. Then $S$ is linearly independent if and only if $S$ spans $V$.  
Contributed by Robert Beezer

T15 Suppose that $A$ is an $m\times n$ matrix and let $\text{ min}\left(m,n\right)$ denote the minimum of $m$ and $n$. Prove that $r\left(A\right)\le \text{ min}\left(m,n\right)$.  
Contributed by Robert Beezer

T20 Suppose that $A$ is an $m\times n$ matrix and $b\in {ℂ}^m$. Prove that the linear system $\mathcal{ℒ}\mathcal{S}\left(A,b\right)$ is consistent if and only if $r\left(A\right)=r\left(\left[\left.A\right|b\right]\right)$.  
Contributed by Robert Beezer Solution [993]

T25 Suppose that $V$ is a vector space with finite dimension. Let $W$ be any subspace of $V$. Prove that $W$ has finite dimension.  
Contributed by Robert Beezer

T60 Suppose that $W$ is a vector space with dimension 5, and $U$ and $V$ are subspaces of $W$, each of dimension 3. Prove that $U\cap V$ contains a non-zero vector. State a more general result.  
Contributed by Joe Riegsecker Solution [993]

Subsection SOL: Solutions

T20 Contributed by Robert Beezer Statement [992]
($\Rightarrow$) Suppose first that $\mathcal{ℒ}\mathcal{S}\left(A,b\right)$ is consistent. Then by Theorem CSCS, $b\in \mathcal{C}\left(A\right)$. This means that $\mathcal{C}\left(A\right)=\mathcal{C}\left(\left[\left.A\right|b\right]\right)$ and so it follows that $r\left(A\right)=r\left(\left[\left.A\right|b\right]\right)$.

($\Leftarrow$) Adding a column to a matrix will only increase the size of its column space, so in all cases, $\mathcal{C}\left(A\right)\subseteq \mathcal{C}\left(\left[\left.A\right|b\right]\right)$. However, if we assume that $r\left(A\right)=r\left(\left[\left.A\right|b\right]\right)$, then by Theorem EDYES we conclude that $\mathcal{C}\left(A\right)=\mathcal{C}\left(\left[\left.A\right|b\right]\right)$. Then $b\in \mathcal{C}\left(\left[\left.A\right|b\right]\right)=\mathcal{C}\left(A\right)$ so by Theorem CSCS, $\mathcal{ℒ}\mathcal{S}\left(A,b\right)$ is consistent.

T60 Contributed by Robert Beezer Statement [992]
Let $\left\{u_1,u_2,u_3\right\}$ and $\left\{v_1,v_2,v_3\right\}$ be bases for $U$ and $V$ (respectively). Then, the set $\left\{u_1,u_2,u_3,v_1,v_2,v_3\right\}$ is linearly dependent, since Theorem G says we cannot have 6 linearly independent vectors in a vector space of dimension 5. So we can assert that there is a non-trivial relation of linear dependence,

where $a_1,a_2,a_3$ and $b_1,b_2,b_3$ are not all zero.

We can rearrange this equation as

This is an equality of two vectors, so we can give this common vector a name, say $w$,

This is the desired non-zero vector, as we will now show.

First, since $w=a_1{u}_1+a_2{u}_2+a_3{u}_3$, we can see that $w\in U$. Similarly, $w=-b_1{v}_1-b_2{v}_2-b_3{v}_3$, so $w\in V$. This establishes that $w\in U\cap V$ (Definition SI).

Is $w\ne 0$? Suppose not, in other words, suppose $w=0$. Then

Because $\left\{u_1,u_2,u_3\right\}$ is a basis for $U$, it is a linearly independent set and the relation of linear dependence above means we must conclude that $a_1=a_2=a_3=0$. By a similar process, we would conclude that $b_1=b_2=b_3=0$. But this is a contradiction since $a_1,a_2,a_3,b_1,b_2,b_3$ were chosen so that some were nonzero. So $w\ne 0$.

How does this generalize? All we really needed was the original relation of linear dependence that resulted because we had “too many” vectors in $W$. A more general statement would be: Suppose that $W$ is a vector space with dimension $n$, $U$ is a subspace of dimension $p$ and $V$ is a subspace of dimension $q$. If $p+q>n$, then $U\cap V$ contains a non-zero vector.