Section SS  Spanning Sets

From A First Course in Linear Algebra
Version 2.12
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/

In this section we will describe a compact way to indicate the elements of an infinite set of vectors, making use of linear combinations. This will give us a convenient way to describe the elements of a set of solutions to a linear system, or the elements of the null space of a matrix, or many other sets of vectors.

Subsection SSV: Span of a Set of Vectors

In Example VFSAL we saw the solution set of a homogeneous system described as all possible linear combinations of two particular vectors. This happens to be a useful way to construct or describe infinite sets of vectors, so we encapsulate this idea in a definition.

Definition SSCV
Span of a Set of Column Vectors
Given a set of vectors S=u1u2u3…up , their span, S , is the set of all possible linear combinations of u1u2u3…up. Symbolically,

S =α1u1+α2u2+α3u3+⋯+αpup∣αi∈ℂ1≤i≤p =∑pi=1αiui∣αi∈ℂ1≤i≤p  

(This definition contains Notation SSV.) △

The span is just a set of vectors, though in all but one situation it is an infinite set. (Just when is it not infinite?) So we start with a finite collection of vectors S (p of them to be precise), and use this finite set to describe an infinite set of vectors, S . Confusing the finite set S with the infinite set S  is one of the most pervasive problems in understanding introductory linear algebra. We will see this construction repeatedly, so let’s work through some examples to get comfortable with it. The most obvious question about a set is if a particular item of the correct type is in the set, or not.

Example ABS
A basic span
Consider the set of 5 vectors, S, from ℂ4

S=                1 1 3 1              2 1 2 −1              7 3 5 −5              1 1 −1 2              −1 0 9 0

and consider the infinite set of vectors S  formed from all possible linear combinations of the elements of S. Here are four vectors we definitely know are elements of S , since we will construct them in accordance with Definition SSCV,

w=(2)      1 1 3 1        +(1)      2 1 2 −1        +(−1)      7 3 5 −5        +(2)      1 1 −1 2        +(3)      −1 0 9 0        =      −4 2 28 10

x=(5)      1 1 3 1        +(−6)      2 1 2 −1        +(−3)      7 3 5 −5        +(4)      1 1 −1 2        +(2)      −1 0 9 0        =      −26 −6 2 34

y=(1)      1 1 3 1        +(0)      2 1 2 −1        +(1)      7 3 5 −5        +(0)      1 1 −1 2        +(1)      −1 0 9 0        =      7 4 17 −4

z=(0)      1 1 3 1        +(0)      2 1 2 −1        +(0)      7 3 5 −5        +(0)      1 1 −1 2        +(0)      −1 0 9 0        =      0 0 0 0

The purpose of a set is to collect objects with some common property, and to exclude objects without that property. So the most fundamental question about a set is if a given object is an element of the set or not. Let’s learn more about S  by investigating which vectors are elements of the set, and which are not.

First, is u=      −15 −6 19 5         an element of S ? We are asking if there are scalars α1α2α3α4α5 such that

α1      1 1 3 1        +α2      2 1 2 −1        +α3      7 3 5 −5        +α4      1 1 −1 2        +α5      −1 0 9 0        =u=      −15 −6 19 5

Applying Theorem SLSLC we recognize the search for these scalars as a solution to a linear system of equations with augmented matrix

1 1 3 1 2 1 2 −1 7 3 5 −5 1 1 −1 2 −1 0 9 0 −15 −6 19 5

which row-reduces to

1 0 0 0 0 1  0 0 −1 4 0 0 0 0 1 0 3 −1 −2 0 10 −9 −7 0

At this point, we see that the system is consistent (Theorem RCLS), so we know there is a solution for the five scalars α1α2α3α4α5. This is enough evidence for us to say that u∈S . If we wished further evidence, we could compute an actual solution, say

This particular solution allows us to write

(2)      1 1 3 1        +(1)      2 1 2 −1        +(−2)      7 3 5 −5        +(−3)      1 1 −1 2        +(2)      −1 0 9 0        =u=      −15 −6 19 5

making it even more obvious that u∈S .

Lets do it again. Is v=      3 1 2 −1         an element of S ? We are asking if there are scalars α1α2α3α4α5 such that

α1      1 1 3 1        +α2      2 1 2 −1        +α3      7 3 5 −5        +α4      1 1 −1 2        +α5      −1 0 9 0        =v=      3 1 2 −1

Applying Theorem SLSLC we recognize the search for these scalars as a solution to a linear system of equations with augmented matrix

1 1 3 1 2 1 2 −1 7 3 5 −5 1 1 −1 2 −1 0 9 0 3 1 2 −1

which row-reduces to

1 0 0 0 0 1  0 0 −1 4 0 0 0 0 1 0 3 −1 −2 0 0 0 0 1

At this point, we see that the system is inconsistent by Theorem RCLS, so we know there is not a solution for the five scalars α1α2α3α4α5. This is enough evidence for us to say that v∉S . End of story. ⊠

Example SCAA
Span of the columns of Archetype A
Begin with the finite set of three vectors of size 3

S=u1u2u3=       1 2 1        −1 1 1        2 1 0

and consider the infinite set S . The vectors of S could have been chosen to be anything, but for reasons that will become clear later, we have chosen the three columns of the coefficient matrix in Archetype A. First, as an example, note that

v=(5)   1 2 1     +(−3)   −1 1 1     +(7)   2 1 0     =   22 14 2

is in S , since it is a linear combination of u1u2u3. We write this succinctly as v∈S . There is nothing magical about the scalars α1=5α2=−3α3=7, they could have been chosen to be anything. So repeat this part of the example yourself, using different values of α1α2α3. What happens if you choose all three scalars to be zero?

So we know how to quickly construct sample elements of the set S . A slightly different question arises when you are handed a vector of the correct size and asked if it is an element of S . For example, is w=   1 8 5      in S ? More succinctly, w∈S ?

To answer this question, we will look for scalars α1α2α3 so that

α1u1+α2u2+α3u3 α1−α2+2α3 2α1+α2+α3 α1+α2    1 0 0 0 1  0 1 −1 0 3 2 0      =w =1 =8 =5 By Theorem SLSLC solutions to this vector equation are solutions to the system of equations Building the augmented matrix for this linear system, and row-reducing, gives  

This system has infinitely many solutions (there’s a free variable in x3), but all we need is one solution vector. The solution,

tells us that

so we are convinced that w really is in S . Notice that there are an infinite number of ways to answer this question affirmatively. We could choose a different solution, this time choosing the free variable to be zero,

shows us that

Verifying the arithmetic in this second solution will make it obvious that w is in this span. And of course, we now realize that there are an infinite number of ways to realize w as element of S . Let’s ask the same type of question again, but this time with y=   2 4 3     , i.e. is y∈S ?

So we’ll look for scalars α1α2α3 so that

α1u1+α2u2+α3u3 α1−α2+2α3 2α1+α2+α3 α1+α2    1 0 0 0 1  0 1 −1 0 0 0 1      =y =2 =4 =3 By Theorem SLSLC solutions to this vector equation are the solutions to the system of equations Building the augmented matrix for this linear system, and row-reducing, gives  

This system is inconsistent (there’s a leading 1 in the last column, Theorem RCLS), so there are no scalars α1α2α3 that will create a linear combination of u1u2u3 that equals y. More precisely, y∉S .

There are three things to observe in this example. (1) It is easy to construct vectors in S . (2) It is possible that some vectors are in S  (e.g. w), while others are not (e.g. y). (3) Deciding if a given vector is in S  leads to solving a linear system of equations and asking if the system is consistent.

With a computer program in hand to solve systems of linear equations, could you create a program to decide if a vector was, or wasn’t, in the span of a given set of vectors? Is this art or science?

This example was built on vectors from the columns of the coefficient matrix of Archetype A. Study the determination that v∈S  and see if you can connect it with some of the other properties of Archetype A. ⊠

Having analyzed Archetype A in Example SCAA, we will of course subject Archetype B to a similar investigation.

Example SCAB
Span of the columns of Archetype B
Begin with the finite set of three vectors of size 3 that are the columns of the coefficient matrix in Archetype B,

R=v1v2v3=       −7 5 1        −6 5 0        −12 7 4

and consider the infinite set R . First, as an example, note that

x=(2)   −7 5 1     +(4)   −6 5 0     +(−3)   −12 7 4     =   −2 9 −10

is in R , since it is a linear combination of v1v2v3. In other words, x∈R . Try some different values of α1α2α3 yourself, and see what vectors you can create as elements of R .

Now ask if a given vector is an element of R . For example, is z=   −33 24 5      in R ? Is z∈R ?

To answer this question, we will look for scalars α1α2α3 so that

α1v1+α2v2+α3v3 −7α1−6α2−12α3 5α1+5α2+7α3 α1+4α3    1 0 0 0 1  0 0 0 1 −3 5 2      =z =−33 =24 =5 By Theorem SLSLC solutions to this vector equation are the solutions to the system of equations Building the augmented matrix for this linear system, and row-reducing, gives  

This system has a unique solution,

telling us that

so we are convinced that z really is in R . Notice that in this case we have only one way to answer the question affirmatively since the solution is unique.

Let’s ask about another vector, say is x=   −7 8 −3     in R ? Is x∈R ?

We desire scalars α1α2α3 so that

α1v1+α2v2+α3v3 −7α1−6α2−12α3 5α1+5α2+7α3 α1+4α3    1 0 0 0 1  0 0 0 1 1 2 −1      =x =−7 =8 =−3 By Theorem SLSLC solutions to this vector equation are the solutions to the system of equations Building the augmented matrix for this linear system, and row-reducing, gives  

This system has a unique solution,

telling us that

so we are convinced that x really is in R . Notice that in this case we again have only one way to answer the question affirmatively since the solution is again unique.

We could continue to test other vectors for membership in R , but there is no point. A question about membership in R  inevitably leads to a system of three equations in the three variables α1α2α3 with a coefficient matrix whose columns are the vectors v1v2v3. This particular coefficient matrix is nonsingular, so by Theorem NMUS, the system is guaranteed to have a solution. (This solution is unique, but that’s not critical here.) So no matter which vector we might have chosen for z, we would have been certain to discover that it was an element of R . Stated differently, every vector of size 3 is in R , or R=ℂ3 .

Compare this example with Example SCAA, and see if you can connect z with some aspects of the write-up for Archetype B. ⊠

Subsection SSNS: Spanning Sets of Null Spaces

We saw in Example VFSAL that when a system of equations is homogeneous the solution set can be expressed in the form described by Theorem VFSLS where the vector c is the zero vector. We can essentially ignore this vector, so that the remainder of the typical expression for a solution looks like an arbitrary linear combination, where the scalars are the free variables and the vectors are u1u2u3…un−r. Which sounds a lot like a span. This is the substance of the next theorem.

Theorem SSNS
Spanning Sets for Null Spaces
Suppose that A is an m×n matrix, and B is a row-equivalent matrix in reduced row-echelon form with r nonzero rows. Let D=d1d2d3…dr  be the column indices where B has leading 1’s (pivot columns) and F=f1f2f3…fn−r  be the set of column indices where B does not have leading 1’s. Construct the n−r vectors zj, 1≤j≤n−r of size n as

zji=        1 0 −Bkfj if i∈F, i=fj if i∈F, i≠fj if i∈D, i=dk

Then the null space of A is given by

NA=z1z2z3…zn−r

Proof   Consider the homogeneous system with A as a coefficient matrix, ℒSA0 . Its set of solutions, S, is by Definition NSM, the null space of A, NA . Let B′ denote the result of row-reducing the augmented matrix of this homogeneous system. Since the system is homogeneous, the final column of the augmented matrix will be all zeros, and after any number of row operations (Definition RO), the column will still be all zeros. So B′ has a final column that is totally zeros.

Now apply Theorem VFSLS to B′, after noting that our homogeneous system must be consistent (Theorem HSC). The vector c has zeros for each entry that corresponds to an index in F. For entries that correspond to an index in D, the value is −B′kn+1 , but for B′ any entry in the final column (index n+1) is zero. So c=0. The vectors zj, 1≤j≤n−r are identical to the vectors uj, 1≤j≤n−r described in Theorem VFSLS. Putting it all together and applying Definition SSCV in the final step,

NA =S =c+α1u1+α2u2+α3u3+⋯+αn−run−r∣α1α2α3…αn−r∈ℂ =α1u1+α2u2+α3u3+⋯+αn−run−r∣α1α2α3…αn−r∈ℂ =z1z2z3…zn−r  

Example SSNS
Spanning set of a null space
Find a set of vectors, S, so that the null space of the matrix A below is the span of S, that is, S=NA .

A=      1 2 1 −1 3 5 1 −4 3 7 5 −2 −1 1 1 0 −5 1 5 4

The null space of A is the set of all solutions to the homogeneous system ℒSA0 . If we find the vector form of the solutions to this homogeneous system (Theorem VFSLS) then the vectors uj, 1≤j≤n−r in the linear combination are exactly the vectors zj, 1≤j≤n−r described in Theorem SSNS. So we can mimic Example VFSAL to arrive at these vectors (rather than being a slave to the formulas in the statement of the theorem).

Begin by row-reducing A. The result is

1 0 0 0 0 1  0 0 6 −1 0 0 0 0 1 0 4 −2 3 0

With D=124  and F=35  we recognize that x3 and x5 are free variables and we can express each nonzero row as an expression for the dependent variables x1, x2, x4 (respectively) in the free variables x3 and x5. With this we can write the vector form of a solution vector as

x1 x2 x3 x4 x5          =        −6x3−4x5 x3+2x5 x3 −3x5 x5          =x3        −6 1 1 0 0          +x5        −4 2 0 −3 1

Then in the notation of Theorem SSNS,

z1 =        −6 1 1 0 0           z2 =        −4 2 0 −3 1            

and

NA=z1z2=                        −6 1 1 0 0                  −4 2 0 −3 1

Example NSDS
Null space directly as a span
Let’s express the null space of A as the span of a set of vectors, applying Theorem SSNS as economically as possible, without reference to the underlying homogeneous system of equations (in contrast to Example SSNS).

A=        2 1 −1 −3 3 1 1 1 2 −1 5 3 −1 −4 5 1 1 0 −4 2 5 6 4 −7 2 1 −1 −3 0 3

Theorem SSNS creates vectors for the span by first row-reducing the matrix in question. The row-reduced version of A is

B=        1 0 0 0 0 0 1  0 0 0 2 1 0 0 0 0 0 1 0 0 −1 3 4 0 0 2 −1 −2 0 0

We will mechanically follow the prescription of Theorem SSNS. Here we go, in two big steps.

First, the non-pivot columns have indices F=356 , so we will construct the n−r=6−3=3 vectors with a pattern of zeros and ones corresponding to the indices in F. This is the realization of the first two lines of the three-case definition of the vectors zj, 1≤j≤n−r.

z1 =   1 0 0      z2 =   0 1 0      z3 =   0 0 1