Section D  Dimension

From A First Course in Linear Algebra
Version 2.00
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/

Almost every vector space we have encountered has been infinite in size (an exception is Example VSS). But some are bigger and richer than others. Dimension, once suitably defined, will be a measure of the size of a vector space, and a useful tool for studying its properties. You probably already have a rough notion of what a mathematical definition of dimension might be — try to forget these imprecise ideas and go with the new ones given here.

Subsection D: Dimension

Definition D
Dimension
Suppose that V is a vector space and v1,v2,v3,,vt is a basis of V . Then the dimension of V is defined by dim V = t. If V has no finite bases, we say V has infinite dimension.

(This definition contains Notation D.)

This is a very simple definition, which belies its power. Grab a basis, any basis, and count up the number of vectors it contains. That’s the dimension. However, this simplicity causes a problem. Given a vector space, you and I could each construct different bases — remember that a vector space might have many bases. And what if your basis and my basis had different sizes? Applying Definition D we would arrive at different numbers! With our current knowledge about vector spaces, we would have to say that dimension is not “well-defined.” Fortunately, there is a theorem that will correct this problem.

In a strictly logical progression, the next two theorems would precede the definition of dimension. Many subsequent theorems will trace their lineage back to the following fundamental result.

Theorem SSLD
Spanning Sets and Linear Dependence
Suppose that S = v1,v2,v3,,vt is a finite set of vectors which spans the vector space V . Then any set of t + 1 or more vectors from V is linearly dependent.

Proof   We want to prove that any set of t + 1 or more vectors from V is linearly dependent. So we will begin with a totally arbitrary set of vectors from V , R = u1,u2,u3,,um, where m > t. We will now construct a nontrivial relation of linear dependence on R.

Each vector u1,u2,u3,,um can be written as a linear combination of v1,v2,v3,,vt since S is a spanning set of V . This means there exist scalars aij, 1 i t, 1 j m, so that

u1 = a11v1 + a21v2 + a31v3 + + at1vt u2 = a12v1 + a22v2 + a32v3 + + at2vt u3 = a13v1 + a23v2 + a33v3 + + at3vt um = a1mv1 + a2mv2 + a3mv3 + + atmvt

Now we form, unmotivated, the homogeneous system of t equations in the m variables, x1,x2,x3,,xm, where the coefficients are the just-discovered scalars aij,

a11x1 + a12x2 + a13x3 + + a1mxm = 0 a21x1 + a22x2 + a23x3 + + a2mxm = 0 a31x1 + a32x2 + a33x3 + + a3mxm = 0 at1x1 + at2x2 + at3x3 + + atmxm = 0

This is a homogeneous system with more variables than equations (our hypothesis is expressed as m > t), so by Theorem HMVEI there are infinitely many solutions. Choose a nontrivial solution and denote it by x1 = c1,x2 = c2,x3 = c3,,xm = cm. As a solution to the homogeneous system, we then have

a11c1 + a12c2 + a13c3 + + a1mcm = 0 a21c1 + a22c2 + a23c3 + + a2mcm = 0 a31c1 + a32c2 + a33c3 + + a3mcm = 0 at1c1 + at2c2 + at3c3 + + atmcm = 0

As a collection of nontrivial scalars, c1,c2,c3,,cm will provide the nontrivial relation of linear dependence we desire,

c1u1 + c2u2 + c3u3 + + cmum = c1 a11v1 + a21v2 + a31v3 + + at1vt  Definition TSVS + c2 a12v1 + a22v2 + a32v3 + + at2vt + c3 a13v1 + a23v2 + a33v3 + + at3vt + cm a1mv1 + a2mv2 + a3mv3 + + atmvt = c1a11v1 + c1a21v2 + c1a31v3 + + c1at1vt  Property DVA + c2a12v1 + c2a22v2 + c2a32v3 + + c2at2vt + c3a13v1 + c3a23v2 + c3a33v3 + + c3at3vt + cma1mv1 + cma2mv2 + cma3mv3 + + cmatmvt = c1a11 + c2a12 + c3a13 + + cma1m v1  Property DSA + c1a21 + c2a22 + c3a23 + + cma2m v2 + c1a31 + c2a32 + c3a33 + + cma3m v3 + c1at1 + c2at2 + c3at3 + + cmatm vt = a11c1 + a12c2 + a13c3 + + a1mcm v1  Property CMCN + a21c1 + a22c2 + a23c3 + + a2mcm v2 + a31c1 + a32c2 + a33c3 + + a3mcm v3 + at1c1 + at2c2 + at3c3 + + atmcm vt = 0v1 + 0v2 + 0v3 + + 0vt  cj as solution = 0 + 0 + 0 + + 0  Theorem ZSSM = 0  Property Z

That does it. R has been undeniably shown to be a linearly dependent set.

The proof just given has some monstrous expressions in it, mostly owing to the double subscripts present. Now is a great opportunity to show the value of a more compact notation. We will rewrite the key steps of the previous proof using summation notation, resulting in a more economical presentation, and even greater insight into the key aspects of the proof. So here is an alternate proof — study it carefully.

Proof   (Alternate Proof of Theorem SSLD) We want to prove that any set of t + 1 or more vectors from V is linearly dependent. So we will begin with a totally arbitrary set of vectors from V , R = uj 1 j m, where m > t. We will now construct a nontrivial relation of linear dependence on R.

Each vector uj, 1 j m can be written as a linear combination of vi, 1 i t since S is a spanning set of V . This means there are scalars aij, 1 i t, 1 j m, so that

uj = i=1ta ijvi 1 j m

Now we form, unmotivated, the homogeneous system of t equations in the m variables, xj, 1 j m, where the coefficients are the just-discovered scalars aij,

j=1ma ijxj = 0 1 i t

This is a homogeneous system with more variables than equations (our hypothesis is expressed as m > t), so by Theorem HMVEI there are infinitely many solutions. Choose one of these solutions that is not trivial and denote it by xj = cj, 1 j m. As a solution to the homogeneous system, we then have j=1ma ijcj = 0 for 1 i t. As a collection of nontrivial scalars, cj, 1 j m, will provide the nontrivial relation of linear dependence we desire,

j=1mc juj = j=1mc j i=1ta ijvi  Definition TSVS = j=1m i=1tc jaijvi  Property DVA = i=1t j=1mc jaijvi  Property CMCN = i=1t j=1ma ijcjvi  Commutativity in  = i=1t j=1ma ijcj vi  Property DSA = i=1t0v i  cj as solution = i=1t0  Theorem ZSSM = 0  Property Z

That does it. R has been undeniably shown to be a linearly dependent set.

Notice how the swap of the two summations is so much easier in the third step above, as opposed to all the rearranging and regrouping that takes place in the previous proof. In about half the space. And there are no ellipses ().

Theorem SSLD can be viewed as a generalization of Theorem MVSLD. We know that m has a basis with m vectors in it (Theorem SUVB), so it is a set of m vectors that spans m. By Theorem SSLD, any set of more than m vectors from m will be linearly dependent. But this is exactly the conclusion we have in Theorem MVSLD. Maybe this is not a total shock, as the proofs of both theorems rely heavily on Theorem HMVEI. The beauty of Theorem SSLD is that it applies in any vector space. We illustrate the generality of this theorem, and hint at its power, in the next example.

Example LDP4
Linearly dependent set in P4
In Example SSP4 we showed that

S = x 2,x2 4x + 4,x3 6x2 + 12x 8,x4 8x3 + 24x2 32x + 16

is a spanning set for W = p(x) p P4,p(2) = 0. So we can apply Theorem SSLD to W with t = 4. Here is a set of five vectors from W, as you may check by verifying that each is a polynomial of degree 4 or less and has x = 2 as a root,

T = p1,p2,p3,p4,p5 W p1 = x4 2x3 + 2x2 8x + 8 p2 = x3 + 6x2 5x 6 p3 = 2x4 5x3 + 5x2 7x + 2 p4 = x4 + 4x3 7x2 + 6x p5 = 4x3 9x2 + 5x 6

By Theorem SSLD we conclude that T is linearly dependent, with no further computations.

Theorem SSLD is indeed powerful, but our main purpose in proving it right now was to make sure that our definition of dimension (Definition D) is well-defined. Here’s the theorem.

Theorem BIS
Bases have Identical Sizes
Suppose that V is a vector space with a finite basis B and a second basis C. Then B and C have the same size.

Proof   Suppose that C has more vectors than B. (Allowing for the possibility that C is infinite, we can replace C by a subset that has more vectors than B.) As a basis, B is a spanning set for V (Definition B), so Theorem SSLD says that C is linearly dependent. However, this contradicts the fact that as a basis C is linearly independent (Definition B). So C must also be a finite set, with size less than, or equal to, that of B.

Suppose that B has more vectors than C. As a basis, C is a spanning set for V (Definition B), so Theorem SSLD says that B is linearly dependent. However, this contradicts the fact that as a basis B is linearly independent (Definition B). So C cannot be strictly smaller than B.

The only possibility left for the sizes of B and C is for them to be equal.

Theorem BIS tells us that if we find one finite basis in a vector space, then they all have the same size. This (finally) makes Definition D unambiguous.

Subsection DVS: Dimension of Vector Spaces

We can now collect the dimension of some common, and not so common, vector spaces.

Theorem DCM
Dimension of m
The dimension of m (Example VSCV) is m.

Proof   Theorem SUVB provides a basis with m vectors.

Theorem DP
Dimension of Pn
The dimension of Pn (Example VSP) is n + 1.

Proof   Example BP provides two bases with n + 1 vectors. Take your pick.

Theorem DM
Dimension of Mmn
The dimension of Mmn (Example VSM) is mn.

Proof   Example BM provides a basis with mn vectors.

Example DSM22
Dimension of a subspace of M22
It should now be plausible that

Z = ab c d 2a + b + 3c + 4d = 0, a + 3b 5c d = 0

is a subspace of the vector space M22 (Example VSM). (It is.) To find the dimension of Z we must first find a basis, though any old basis will do.

First concentrate on the conditions relating a,b,c and d. They form a homogeneous system of two equations in four variables with coefficient matrix

2 1 3 4 1351

We can row-reduce this matrix to obtain

10 2 2 0 110

Rewrite the two equations represented by each row of this matrix, expressing the dependent variables (a and b) in terms of the free variables (c and d), and we obtain,

a = 2c 2d b = c

We can now write a typical entry of Z strictly in terms of c and d, and we can decompose the result,

ab c d = 2c 2dc c d = 2cc c 0 + 2d0 0 d = c 21 1 0 +d 20 0 1

this equation says that an arbitrary matrix in Z can be written as a linear combination of the two vectors in

S = 21 1 0 , 20 0 1

so we know that

Z = S = 21 1 0 , 20 0 1

Are these two matrices (vectors) also linearly independent? Begin with a relation of linear dependence on S,

a1 21 1 0 + a2 20 0 1 = O 2a1 2a2a1 a1 a2 = 00 0 0

From the equality of the two entries in the last row, we conclude that a1 = 0, a2 = 0. Thus the only possible relation of linear dependence is the trivial one, and therefore S is linearly independent (Definition LI). So S is a basis for V (Definition B). Finally, we can conclude that dim Z = 2 (Definition D) since S has two elements.

Example DSP4
Dimension of a subspace of P4
In Example BSP4 we showed that

S = x 2,x2 4x + 4,x3 6x2 + 12x 8,x4 8x3 + 24x2 32x + 16

is a basis for W = p(x) p P4,p(2) = 0. Thus, the dimension of W is four, dim W = 4.

Note that dim P4 = 5 by Theorem DP, so W is a subspace of dimension 4 within the vector space P4 of dimension 5, illustrating the upcoming Theorem PSSD.

Example DC
Dimension of the crazy vector space
In Example BC we determined that the set R = (1,0),(6,3) from the crazy vector space, C (Example CVS), is a basis for C. By Definition D we see that C has dimension 2, dim C = 2.

It is possible for a vector space to have no finite bases, in which case we say it has infinite dimension. Many of the best examples of this are vector spaces of functions, which lead to constructions like Hilbert spaces. We will focus exclusively on finite-dimensional vector spaces. OK, one infinite-dimensional example, and then we will focus exclusively on finite-dimensional vector spaces.

Example VSPUD
Vector space of polynomials with unbounded degree
Define the set P by

P = pp(x)  is a polynomial in x

Our operations will be the same as those defined for Pn (Example VSP).

With no restrictions on the possible degrees of our polynomials, any finite set that is a candidate for spanning P will come up short. We will give a proof by contradiction (Technique CD). To this end, suppose that the dimension of P is finite, say dim P = n.

The set T = 1,x,x2,,xn is a linearly independent set (check this!) containing n + 1 polynomials from P. However, a basis of P will be a spanning set of P containing n vectors. This situation is a contradiction of Theorem SSLD, so our assumption that P has finite dimension is false. Thus, we say dim P = .

Subsection RNM: Rank and Nullity of a Matrix

For any matrix, we have seen that we can associate several subspaces — the null space (Theorem NSMS), the column space (Theorem CSMS), row space (Theorem RSMS) and the left null space (Theorem LNSMS). As vector spaces, each of these has a dimension, and for the null space and column space, they are important enough to warrant names.

Definition NOM
Nullity Of a Matrix
Suppose that A is an m × n matrix. Then the nullity of A is the dimension of the null space of A, n A = dim NA.

(This definition contains Notation NOM.)

Definition ROM
Rank Of a Matrix
Suppose that A is an m × n matrix. Then the rank of A is the dimension of the column space of A, r A = dim CA.

(This definition contains Notation ROM.)

Example RNM
Rank and nullity of a matrix
Let’s compute the rank and nullity of

A = 2 41 3 2 1 4 1 2 0 0 4 0 1 2 4 1 0 548 1 2 1 1 6 1 3 2 41 1 4 21 1 2 3 1 6 3 1

To do this, we will first row-reduce the matrix since that will help us determine bases for the null space and column space.

1200 4 0 1 0 0 10 3 02 0 0 01103 0 0 00 0 1 1 0 0 00 0 0 0 0 0 00 0 0 0

From this row-equivalent matrix in reduced row-echelon form we record D = 1,3,4,6 and F = 2,5,7.

For each index in D, Theorem BCS creates a single basis vector. In total the basis will have 4 vectors, so the column space of A will have dimension 4 and we write r A = 4.

For each index in F, Theorem BNS creates a single basis vector. In total the basis will have 3 vectors, so the null space of A will have dimension 3 and we write n A = 3.

There were no accidents or coincidences in the previous example — with the row-reduced version of a matrix in hand, the rank and nullity are easy to compute.

Theorem CRN
Computing Rank and Nullity
Suppose that A is an m × n matrix and B is a row-equivalent matrix in reduced row-echelon form with r nonzero rows. Then r A = r and n A = n r.

Proof   Theorem BCS provides a basis for the column space by choosing columns of A that correspond to the dependent variables in a description of the solutions to SA,0. In the analysis of B, there is one dependent variable for each leading 1, one per nonzero row, or one per pivot column. So there are r column vectors in a basis for CA.

Theorem BNS provide a basis for the null space by creating basis vectors of the null space of A from entries of B, one for each independent variable, one per column with out a leading 1. So there are n r column vectors in a basis for n A.

Every archetype (Appendix A) that involves a matrix lists its rank and nullity. You may have noticed as you studied the archetypes that the larger the column space is the smaller the null space is. A simple corollary states this trade-off succinctly. (See Technique LC.)

Theorem RPNC
Rank Plus Nullity is Columns
Suppose that A is an m × n matrix. Then r A + n A = n.

Proof   Let r be the number of nonzero rows in a row-equivalent matrix in reduced row-echelon form. By Theorem CRN,

r A + n A = r + (n r) = n

When we first introduced r as our standard notation for the number of nonzero rows in a matrix in reduced row-echelon form you might have thought r stood for “rows.” Not really — it stands for “rank”!

Subsection RNNM: Rank and Nullity of a Nonsingular Matrix

Let’s take a look at the rank and nullity of a square matrix.

Example RNSM
Rank and nullity of a square matrix
The matrix

E = 0 4 1 2 2 3 1 2 2 1 1 0 43 23 9 3 9 1 9 34 9 4 1 6 2 34 6 2 5 9 4 9 3 8 24 2 4 8 2 2 9 3 0 9

is row-equivalent to the matrix in reduced row-echelon form,

1000000 0 100000 0 010000 0 001000 0 000100 0 000010 0 000001

With n = 7 columns and r = 7 nonzero rows Theorem CRN tells us the rank is r E = 7 and the nullity is n E = 7 7 = 0.

The value of either the nullity or the rank are enough to characterize a nonsingular matrix.

Theorem RNNM
Rank and Nullity of a Nonsingular Matrix
Suppose that A is a square matrix of size n. The following are equivalent.

  1. A is nonsingular.
  2. The rank of A is n, r A = n.
  3. The nullity of A is zero, n A = 0.

Proof   (1 2) Theorem CSNM says that if A is nonsingular then CA = n. If CA = n, then the column space has dimension n by Theorem DCM, so the rank of A is n.
(2 3) Suppose r A = n. Then Theorem RPNC gives

n A = n r A  Theorem RPNC = n n  Hypothesis = 0

(3 1) Suppose n A = 0, so a basis for the null space of A is the empty set. This implies that NA = 0 and Theorem NMTNS says A is nonsingular.

With a new equivalence for a nonsingular matrix, we can update our list of equivalences (Theorem NME5) which now becomes a list requiring double digits to number.

Theorem NME6
Nonsingular Matrix Equivalences, Round 6
Suppose that A is a square matrix of size n. The following are equivalent.

  1. A is nonsingular.
  2. A row-reduces to the identity matrix.
  3. The null space of A contains only the zero vector, NA = 0.
  4. The linear system SA,b has a unique solution for every possible choice of b.
  5. The columns of A are a linearly independent set.
  6. A is invertible.
  7. The column space of A is n, CA = n.
  8. The columns of A are a basis for n.
  9. The rank of A is n, r A = n.
  10. The nullity of A is zero, n A = 0.

Proof   Building on Theorem NME5 we can add two of the statements from Theorem RNNM.

Subsection READ: Reading Questions

  1. What is the dimension of the vector space P6, the set of all polynomials of degree 6 or less?
  2. How are the rank and nullity of a matrix related?
  3. Explain why we might say that a nonsingular matrix has “full rank.”

Subsection EXC: Exercises

C20 The archetypes listed below are matrices, or systems of equations with coefficient matrices. For each, compute the nullity and rank of the matrix. This information is listed for each archetype (along with the number of columns in the matrix, so as to illustrate Theorem RPNC), and notice how it could have been computed immediately after the determination of the sets D and F associated with the reduced row-echelon form of the matrix.
Archetype A
Archetype B
Archetype C
Archetype D/Archetype E
Archetype F
Archetype G/Archetype H
Archetype I
Archetype J
Archetype K
Archetype L  
Contributed by Robert Beezer

C30 For the matrix A below, compute the dimension of the null space of A, dim NA.

A = 21311 9 1 2 1 73 3 1 3 6 8 2 1 2 53

 
Contributed by Robert Beezer Solution [1020]

C31 The set W below is a subspace of 4. Find the dimension of W.

W = 2 3 4 1 , 3 0 1 2 , 4 3 2 5

 
Contributed by Robert Beezer Solution [1021]

C40 In Example LDP4 we determined that the set of five polynomials, T, is linearly dependent by a simple invocation of Theorem SSLD. Prove that T is linearly dependent from scratch, beginning with Definition LI.  
Contributed by Robert Beezer

M20 M22 is the vector space of 2 × 2 matrices. Let S22 denote the set of all 2 × 2 symmetric matrices. That is

S22 = A M22 At = A

(a) Show that S22 is a subspace of M22.
(b) Exhibit a basis for S22 and prove that it has the required properties.
(c) What is the dimension of S22?  
Contributed by Robert Beezer Solution [1022]

M21 A 2 × 2 matrix B is upper triangular if B21 = 0. Let UT2 be the set of all 2 × 2 upper triangular matrices. Then UT2 is a subspace of the vector space of all 2 × 2 matrices, M22 (you may assume this). Determine the dimension of UT2 providing all of the necessary justifications for your answer.  
Contributed by Robert Beezer Solution [1025]

Subsection SOL: Solutions

C30 Contributed by Robert Beezer Statement [1017]
Row reduce A,

A  RREF 100 1 1 0 1031 0 0122 0 00 0 0

So r = 3 for this matrix. Then

dim NA = n A  Definition NOM = n A + r A r A = 5 r A  Theorem RPNC = 5 3  Theorem CRN = 2

We could also use Theorem BNS and create a basis for NA with n r = 5 3 = 2 vectors (because the solutions are described with 2 free variables) and arrive at the dimension as the size of this basis.

C31 Contributed by Robert Beezer Statement [1018]
We will appeal to Theorem BS (or you could consider this an appeal to Theorem BCS). Put the three columnn vectors of this spanning set into a matrix as columns and row-reduce.

2 3 4 3 0 3 4 1 2 1 2 5  RREF 10 1 0 12 0 0 0 0 0 0

The pivot columns are D = 1, 2 so we can “keep” the vectors corresponding to the pivot columns and set

T = 2 3 4 1 , 3 0 1 2

and conclude that W = T and T is linearly independent. In other words, T is a basis with two vectors, so W has dimension 2.

M20 Contributed by Robert Beezer Statement [1019]
(a) We will use the three criteria of Theorem TSS. The zero vector of M22 is the zero matrix, O (Definition ZM), which is a symmetric matrix. So S22 is not empty, since O S22.

Suppose that A and B are two matrices in S22. Then we know that At = A and Bt = B. We want to know if A + B S22, so test A + B for membership,

A + Bt = At + Bt  Theorem TMA = A + B A,B S22

So A + B is symmetric and qualifies for membership in S22.

Suppose that A S22 and α . Is αA S22? We know that At = A. Now check that,

αAt = αAt  Theorem TMSM = αA A S22

So αA is also symmetric and qualifies for membership in S22.

With the three criteria of Theorem TSS fulfilled, we see that S22 is a subspace of M22.

(b) An arbitrary matrix from S22 can be written as ab b d . We can express this matrix as

ab b d = a0 0 0 + 0b b 0 + 00 0 d = a 10 0 0 + b 01 1 0 + d 00 0 1

this equation says that the set

T = 10 0 0 , 01 1 0 , 00 0 1

spans S22. Is it also linearly independent?

Write a relation of linear dependence on S,

O = a1 10 0 0 + a2 01 1 0 + a3 00 0 1 00 0 0 = a1a2 a2 a3

The equality of these two matrices (Definition ME) tells us that a1 = a2 = a3 = 0, and the only relation of linear dependence on T is trivial. So T is linearly independent, and hence is a basis of S22.

(c) The basis T found in part (b) has size 3. So by Definition D, dim S22 = 3.

M21 Contributed by Robert Beezer Statement [1019]
A typical matrix from UT2 looks like

ab 0 c

where a,b,c are arbitrary scalars. Observing this we can then write

ab 0 c = a 10 0 0 +b 01 0 0 +c 00 0 1

which says that

R = 10 0 0 , 01 0 0 , 00 0 1

is a spanning set for UT2 (Definition TSVS). Is R is linearly independent? If so, it is a basis for UT2. So consider a relation of linear dependence on R,

α1 10 0 0 +α2 01 0 0 +α3 00 0 1 = O = 00 0 0

From this equation, one rapidly arrives at the conclusion that α1 = α2 = α3 = 0. So R is a linearly independent set (Definition LI), and hence is a basis (Definition B) for UT2. Now, we simply count up the size of the set R to see that the dimension of UT2 is dim UT2 = 3.