From A First Course in Linear Algebra
Version 2.20
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/
In this section we will describe a compact way to indicate the elements of an
infinite set of vectors, making use of linear combinations. This will give us a
convenient way to describe the elements of a set of solutions to a linear
system, or the elements of the null space of a matrix, or many other sets of
vectors.
In Example VFSAL we saw the solution set of a homogeneous system described as all possible linear combinations of two particular vectors. This happens to be a useful way to construct or describe infinite sets of vectors, so we encapsulate this idea in a definition.
Definition SSCV
Span of a Set of Column Vectors
Given a set of vectors ,
their span, ,
is the set of all possible linear combinations of
.
Symbolically,
(This definition contains Notation SSV.)
The span is just a set of vectors, though in all but one situation it is an infinite set. (Just when is it not infinite?) So we start with a finite collection of vectors ( of them to be precise), and use this finite set to describe an infinite set of vectors, . Confusing the finite set with the infinite set is one of the most pervasive problems in understanding introductory linear algebra. We will see this construction repeatedly, so let’s work through some examples to get comfortable with it. The most obvious question about a set is if a particular item of the correct type is in the set, or not.
Example ABS
A basic span
Consider the set of 5 vectors, ,
from
and consider the infinite set of vectors formed from all possible linear combinations of the elements of . Here are four vectors we definitely know are elements of , since we will construct them in accordance with Definition SSCV,
The purpose of a set is to collect objects with some common property, and to exclude objects without that property. So the most fundamental question about a set is if a given object is an element of the set or not. Let’s learn more about by investigating which vectors are elements of the set, and which are not.
First, is an element of ? We are asking if there are scalars such that
Applying Theorem SLSLC we recognize the search for these scalars as a solution to a linear system of equations with augmented matrix
which row-reduces to
At this point, we see that the system is consistent (Theorem RCLS), so we know there is a solution for the five scalars . This is enough evidence for us to say that . If we wished further evidence, we could compute an actual solution, say
This particular solution allows us to write
making it even more obvious that .
Lets do it again. Is an element of ? We are asking if there are scalars such that
Applying Theorem SLSLC we recognize the search for these scalars as a solution to a linear system of equations with augmented matrix
which row-reduces to
At this point, we see that the system is inconsistent by Theorem RCLS, so we know there is not a solution for the five scalars . This is enough evidence for us to say that . End of story.
Example SCAA
Span of the columns of Archetype A
Begin with the finite set of three vectors of size
and consider the infinite set . The vectors of could have been chosen to be anything, but for reasons that will become clear later, we have chosen the three columns of the coefficient matrix in Archetype A. First, as an example, note that
is in , since it is a linear combination of . We write this succinctly as . There is nothing magical about the scalars , they could have been chosen to be anything. So repeat this part of the example yourself, using different values of . What happens if you choose all three scalars to be zero?
So we know how to quickly construct sample elements of the set . A slightly different question arises when you are handed a vector of the correct size and asked if it is an element of . For example, is in ? More succinctly, ?
To answer this question, we will look for scalars so that
This system has infinitely many solutions (there’s a free variable in ), but all we need is one solution vector. The solution,
tells us that
so we are convinced that really is in . Notice that there are an infinite number of ways to answer this question affirmatively. We could choose a different solution, this time choosing the free variable to be zero,
shows us that
Verifying the arithmetic in this second solution will make it obvious that is in this span. And of course, we now realize that there are an infinite number of ways to realize as element of . Let’s ask the same type of question again, but this time with , i.e. is ?
So we’ll look for scalars so that
This system is inconsistent (there’s a leading 1 in the last column, Theorem RCLS), so there are no scalars that will create a linear combination of that equals . More precisely, .
There are three things to observe in this example. (1) It is easy to construct vectors in . (2) It is possible that some vectors are in (e.g. ), while others are not (e.g. ). (3) Deciding if a given vector is in leads to solving a linear system of equations and asking if the system is consistent.
With a computer program in hand to solve systems of linear equations, could you create a program to decide if a vector was, or wasn’t, in the span of a given set of vectors? Is this art or science?
This example was built on vectors from the columns of the coefficient matrix of Archetype A. Study the determination that and see if you can connect it with some of the other properties of Archetype A.
Having analyzed Archetype A in Example SCAA, we will of course subject Archetype B to a similar investigation.
Example SCAB
Span of the columns of Archetype B
Begin with the finite set of three vectors of size
that
are the columns of the coefficient matrix in Archetype B,
and consider the infinite set . First, as an example, note that
is in , since it is a linear combination of . In other words, . Try some different values of yourself, and see what vectors you can create as elements of .
Now ask if a given vector is an element of . For example, is in ? Is ?
To answer this question, we will look for scalars so that
This system has a unique solution,
telling us that
so we are convinced that really is in . Notice that in this case we have only one way to answer the question affirmatively since the solution is unique.
Let’s ask about another vector, say is in ? Is ?
We desire scalars so that
This system has a unique solution,
telling us that
so we are convinced that really is in . Notice that in this case we again have only one way to answer the question affirmatively since the solution is again unique.
We could continue to test other vectors for membership in , but there is no point. A question about membership in inevitably leads to a system of three equations in the three variables with a coefficient matrix whose columns are the vectors . This particular coefficient matrix is nonsingular, so by Theorem NMUS, the system is guaranteed to have a solution. (This solution is unique, but that’s not critical here.) So no matter which vector we might have chosen for , we would have been certain to discover that it was an element of . Stated differently, every vector of size 3 is in , or .
Compare this example with Example SCAA, and see if you can connect with some aspects of the write-up for Archetype B.
We saw in Example VFSAL that when a system of equations is homogeneous the solution set can be expressed in the form described by Theorem VFSLS where the vector is the zero vector. We can essentially ignore this vector, so that the remainder of the typical expression for a solution looks like an arbitrary linear combination, where the scalars are the free variables and the vectors are . Which sounds a lot like a span. This is the substance of the next theorem.
Theorem SSNS
Spanning Sets for Null Spaces
Suppose that
is an
matrix, and
is a row-equivalent matrix in reduced row-echelon form with
nonzero rows. Let
be the column indices where
has leading 1’s (pivot columns)
and be the set of column
indices where does not have
leading 1’s. Construct the
vectors ,
of
size
as
Then the null space of is given by
Proof Consider the homogeneous system with as a coefficient matrix, . Its set of solutions, , is by Definition NSM, the null space of , . Let denote the result of row-reducing the augmented matrix of this homogeneous system. Since the system is homogeneous, the final column of the augmented matrix will be all zeros, and after any number of row operations (Definition RO), the column will still be all zeros. So has a final column that is totally zeros.
Now apply Theorem VFSLS to , after noting that our homogeneous system must be consistent (Theorem HSC). The vector has zeros for each entry that corresponds to an index in . For entries that correspond to an index in , the value is , but for any entry in the final column (index ) is zero. So . The vectors , are identical to the vectors , described in Theorem VFSLS. Putting it all together and applying Definition SSCV in the final step,
Example SSNS
Spanning set of a null space
Find a set of vectors, , so that
the null space of the matrix
below is the span of ,
that is, .
The null space of is the set of all solutions to the homogeneous system . If we find the vector form of the solutions to this homogeneous system (Theorem VFSLS) then the vectors , in the linear combination are exactly the vectors , described in Theorem SSNS. So we can mimic Example VFSAL to arrive at these vectors (rather than being a slave to the formulas in the statement of the theorem).
Begin by row-reducing . The result is
With and we recognize that and are free variables and we can express each nonzero row as an expression for the dependent variables , , (respectively) in the free variables and . With this we can write the vector form of a solution vector as
Then in the notation of Theorem SSNS,
and
Example NSDS
Null space directly as a span
Let’s express the null space of
as the span of a set of vectors, applying Theorem SSNS as economically as
possible, without reference to the underlying homogeneous system of equations (in
contrast to Example SSNS).
Theorem SSNS creates vectors for the span by first row-reducing the matrix in question. The row-reduced version of is
We will mechanically follow the prescription of Theorem SSNS. Here we go, in two big steps.
First, the non-pivot columns have indices , so we will construct the vectors with a pattern of zeros and ones corresponding to the indices in . This is the realization of the first two lines of the three-case definition of the vectors , .
Each of these vectors arises due to the presence of a column that is not a pivot column. The remaining entries of each vector are the entries of the corresponding non-pivot column, negated, and distributed into the empty slots in order (these slots have indices in the set and correspond to pivot columns). This is the realization of the third line of the three-case definition of the vectors , .
So, by Theorem SSNS, we have
We know that the null space of is the solution set of the homogeneous system , but nowhere in this application of Theorem SSNS have we found occasion to reference the variables or equations of this system. These details are all buried in the proof of Theorem SSNS.
More advanced computational devices will compute the null space of a matrix. See: Computation NS.MMA Here’s an example that will simultaneously exercise the span construction and Theorem SSNS, while also pointing the way to the next section.
Example SCAD
Span of the columns of Archetype D
Begin with the set of four vectors of size
and consider the infinite set . The vectors of have been chosen as the four columns of the coefficient matrix in Archetype D. Check that the vector
is a solution to the homogeneous system (it is the vector provided by the description of the null space of the coefficient matrix from Theorem SSNS). Applying Theorem SLSLC, we can write the linear combination,
which we can solve for ,
This equation says that whenever we encounter the vector , we can replace it with a specific linear combination of the vectors and . So using in the set , along with and , is excessive. An example of what we mean here can be illustrated by the computation,
So what began as a linear combination of the vectors has been reduced to a linear combination of the vectors . A careful proof using our definition of set equality (Definition SE) would now allow us to conclude that this reduction is possible for any vector in , so
So the span of our set of vectors, , has not changed, but we have described it by the span of a set of three vectors, rather than four. Furthermore, we can achieve yet another, similar, reduction.
Check that the vector
is a solution to the homogeneous system (it is the vector provided by the description of the null space of the coefficient matrix from Theorem SSNS). Applying Theorem SLSLC, we can write the linear combination,
which we can solve for ,
This equation says that whenever we encounter the vector , we can replace it with a specific linear combination of the vectors and . So, as before, the vector is not needed in the description of , provided we have and available. In particular, a careful proof (such as is done in Example RSC5) would show that
So began life as the span of a set of four vectors, and we have now shown (utilizing solutions to a homogeneous system) that can also be described as the span of a set of just two vectors. Convince yourself that we cannot go any further. In other words, it is not possible to dismiss either or in a similar fashion and winnow the set down to just one vector.
What was it about the original set of four vectors that allowed us to declare certain vectors as surplus? And just which vectors were we able to dismiss? And why did we have to stop once we had two vectors remaining? The answers to these questions motivate “linear independence,” our next section and next definition, and so are worth considering carefully now.
It is possible to have your computational device crank out the vector form of the solution set to a linear system of equations. See: Computation VFSS.MMA
Let be the span of S. Is the vector in ? Give an explanation of the reason for your answer.
C22 For each archetype that is a system of equations, consider the
corresponding homogeneous system of equations. Write elements of the
solution set to these homogeneous systems in vector form, as guaranteed by
Theorem VFSLS. Then write the null space of the coefficient matrix of each
system as the span of a set of vectors, as described in Theorem SSNS.
Archetype A
Archetype B
Archetype C
Archetype D/ Archetype E
Archetype F
Archetype G/ Archetype H
Archetype I
Archetype J
Contributed by Robert Beezer Solution [383]
C23 Archetype K and Archetype L are defined as matrices. Use Theorem SSNS directly
to find a set
so that
is the null space of the matrix. Do not make any reference to the associated
homogeneous system of equations in your solution.
Contributed by Robert Beezer Solution [383]
C40 Suppose that .
Let and
let . Is
? If
so, provide an explicit linear combination that demonstrates this.
Contributed by Robert Beezer Solution [383]
C41 Suppose that .
Let and
let . Is
? If
so, provide an explicit linear combination that demonstrates this.
Contributed by Robert Beezer Solution [384]
C42 Suppose .
Is in
?
Contributed by Robert Beezer Solution [386]
C43 Suppose .
Is in
?
Contributed by Robert Beezer Solution [387]
C44 Suppose that .
Let and
let . Is
? If
so, provide an explicit linear combination that demonstrates this.
Contributed by Robert Beezer Solution [389]
C45 Suppose that .
Let and
let . Is
? If
so, provide an explicit linear combination that demonstrates this.
Contributed by Robert Beezer Solution [391]
C50 Let
be the matrix below.
(a) Find a set
so that .
(b) If , then show
directly that .
(c) Write as a linear
combination of the vectors in .
Contributed by Robert Beezer Solution [392]
C60 For the matrix below, find a set of vectors so that the span of equals the null space of , .
Contributed by Robert Beezer Solution [395]
M10 Consider the set of all size vectors in the Cartesian plane .
Contributed by Chris Black Solution [396]
M11 Consider the set of all size vectors in Cartesian 3-space .
Contributed by Chris Black Solution [397]
M12 Let and .
Contributed by Chris Black Solution [398]
M20 In Example SCAD we began with the four columns of the coefficient
matrix of Archetype D, and used these columns in a span construction. Then we
methodically argued that we could remove the last column, then the third
column, and create the same set by just doing a span construction with the first
two columns. We claimed we could not go any further, and had removed as many
vectors as possible. Provide a convincing argument for why a third vector cannot
be removed.
Contributed by Robert Beezer
M21 In the spirit of Example SCAD, begin with the four columns of the coefficient
matrix of Archetype C, and use these columns in a span construction to build the set
. Argue
that
can be expressed as the span of just three of the columns of the coefficient matrix
(saying exactly which three) and in the spirit of Exercise SS.M20 argue that no one
of these three vectors can be removed and still have a span construction create
.
Contributed by Robert Beezer Solution [399]
T10 Suppose that . Prove that
Contributed by Robert Beezer Solution [400]
T20 Suppose that is a
set of vectors from . Prove
that the zero vector, ,
is an element of .
Contributed by Robert Beezer Solution [402]
T21 Suppose that
is a set of vectors from
and . Prove
that .
Contributed by Robert Beezer
T22 Suppose that is
a set of vectors from ,
, and
. Prove
that .
Contributed by Robert Beezer
C22 Contributed by Robert Beezer Statement [377]
The vector form of the solutions obtained in this manner will involve precisely the
vectors described in Theorem SSNS as providing the null space of the coefficient
matrix of the system as a span. These vectors occur in each archetype in a
description of the null space. Studying Example VFSAL may be of some
help.
C23 Contributed by Robert Beezer Statement [377]
Study Example NSDS to understand the correct approach to this question. The
solution for each is listed in the Archetypes (Appendix A) themselves.
C40 Contributed by Robert Beezer Statement [377]
Rephrasing the question, we want to know if there are scalars
and
such
that
Theorem SLSLC allows us to rephrase the question again as a quest for solutions to the system of four equations in two unknowns with an augmented matrix given by
This matrix row-reduces to
From the form of this matrix, we can see that and is an affirmative answer to our question. More convincingly,
C41 Contributed by Robert Beezer Statement [377]
Rephrasing the question, we want to know if there are scalars
and
such
that
Theorem SLSLC allows us to rephrase the question again as a quest for solutions to the system of four equations in two unknowns with an augmented matrix given by
This matrix row-reduces to
With a leading 1 in the last column of this matrix (Theorem RCLS) we can see that the system of equations has no solution, so there are no values for and that will allow us to conclude that is in . So .
C42 Contributed by Robert Beezer Statement [378]
Form a linear combination, with unknown scalars, of
that
equals ,
We want to know if there are values for the scalars that make the vector equation true since that is the definition of membership in . By Theorem SLSLC any such values will also be solutions to the linear system represented by the augmented matrix,
Row-reducing the matrix yields,
From this we see that the system of equations is consistent (Theorem RCLS), and has a unique solution. This solution will provide a linear combination of the vectors in that equals . So .
C43 Contributed by Robert Beezer Statement [378]
Form a linear combination, with unknown scalars, of
that
equals ,
We want to know if there are values for the scalars that make the vector equation true since that is the definition of membership in . By Theorem SLSLC any such values will also be solutions to the linear system represented by the augmented matrix,
Row-reducing the matrix yields,
With a leading 1 in the last column, the system is inconsistent (Theorem RCLS), so there are no scalars that will create a linear combination of the vectors in that equal . So .
C44 Contributed by Robert Beezer Statement [378]
Form a linear combination, with unknown scalars, of
that
equals ,
We want to know if there are values for the scalars that make the vector equation true since that is the definition of membership in . By Theorem SLSLC any such values will also be solutions to the linear system represented by the augmented matrix,
Row-reducing the matrix yields,
From this we see that the system of equations is consistent (Theorem RCLS), and has a infinitely many solutions. Any solution will provide a linear combination of the vectors in that equals . So , for example,
C45 Contributed by Robert Beezer Statement [378]
Form a linear combination, with unknown scalars, of
that
equals ,
We want to know if there are values for the scalars that make the vector equation true since that is the definition of membership in . By Theorem SLSLC any such values will also be solutions to the linear system represented by the augmented matrix,
Row-reducing the matrix yields,
With a leading 1 in the last column, the system is inconsistent (Theorem RCLS), so there are no scalars that will create a linear combination of the vectors in that equal . So .
C50 Contributed by Robert Beezer Statement [378]
(a) Theorem SSNS provides formulas for a set
with this property, but first we must row-reduce
and would be the free variables in the homogeneous system and Theorem SSNS provides the set where
(b) Simply employ the components of the vector as the variables in the homogeneous system . The three equations of this system evaluate as follows,
Since each result is zero, qualifies for membership in .
(c) By Theorem SSNS we know this must be possible (that is the moral of this exercise). Find scalars and so that
Theorem SLSLC allows us to convert this question into a question about a system of four equations in two variables. The augmented matrix of this system row-reduces to
A solution is and . (Notice too that this solution is unique!)
C60 Contributed by Robert Beezer Statement [379]
Theorem SSNS says that if we find the vector form of the solutions to the homogeneous
system ,
then the fixed vectors (one per free variable) will have the desired property. Row-reduce
,
viewing it as the augmented matrix of a homogeneous system with an invisible
columns of zeros as the last column,
Moving to the vector form of the solutions (Theorem VFSLS), with free variables and , solutions to the consistent system (it is homogeneous, Theorem HSC) can be expressed as
Then with given by
Theorem SSNS guarantees that
M10 Contributed by Chris Black Statement [380]
M11 Contributed by Chris Black Statement [380]
M12 Contributed by Chris Black Statement [380]
M21 Contributed by Robert Beezer Statement [381]
If the columns of the coefficient matrix from Archetype C are named
then
we can discover the equation
by building a homogeneous system of equations and viewing a solution to the system as scalars in a linear combination via Theorem SLSLC. This particular vector equation can be rearranged to read
This can be interpreted to mean that is unnecessary in , so that
If we try to repeat this process and find a linear combination of that equals the zero vector, we will fail. The required homogeneous system of equations (via Theorem SLSLC) has only a trivial solution, which will not provide the kind of equation we need to remove one of the three remaining vectors.
T10 Contributed by Robert Beezer Statement [381]
This is an equality of sets, so Definition SE applies.
First show that .
Choose .
Then for
some scalars
and .
Then,
which qualifies for membership in , as it is a linear combination of .
Now show the opposite inclusion, .
Choose . Then
there are scalars
such that
Rearranging, we obtain,
This is an expression for as a linear combination of and , earning membership in . Since is a subset of , and vice versa, we see that , as desired.
T20 Contributed by Robert Beezer Statement [382]
No matter what the elements of the set
are,
we can choose the scalars in a linear combination to all be zero. Suppose that
.
Then compute
But what if we choose to be the empty set? The convention is that the empty sum in Definition SSCV evaluates to “zero,” in this case this is the zero vector.