Section LC  Linear Combinations

From A First Course in Linear Algebra
Version 2.20
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/

In Section VO we defined vector addition and scalar multiplication. These two operations combine nicely to give us a construction known as a linear combination, a construct that we will work with throughout this course.

Subsection LC: Linear Combinations

Definition LCCV
Linear Combination of Column Vectors
Given n vectors u1,u2,u3,,un from m and n scalars α1,α2,α3,,αn, their linear combination is the vector

α1u1 + α2u2 + α3u3 + + αnun

So this definition takes an equal number of scalars and vectors, combines them using our two new operations (scalar multiplication and vector addition) and creates a single brand-new vector, of the same size as the original vectors. When a definition or theorem employs a linear combination, think about the nature of the objects that go into its creation (lists of scalars and vectors), and the type of object that results (a single vector). Computationally, a linear combination is pretty easy.

Example TLC
Two linear combinations in 6
Suppose that

α1 = 1 α2 = 4 α3 = 2 α4 = 1

and

u1 = 2 4 3 1 2 9 u2 = 6 3 0 2 1 4 u3 = 5 2 1 1 3 0 u4 = 3 2 5 7 1 3

then their linear combination is

α1u1 + α2u2 + α3u3 + α4u4 = (1) 2 4 3 1 2 9 + (4) 6 3 0 2 1 4 + (2) 5 2 1 1 3 0 + (1) 3 2 5 7 1 3 = 2 4 3 1 2 9 + 24 12 0 8 4 16 + 10 4 2 2 6 0 + 3 2 5 7 1 3 = 35 6 4 4 9 10 .

A different linear combination, of the same set of vectors, can be formed with different scalars. Take

β1 = 3 β2 = 0 β3 = 5 β4 = 1

and form the linear combination

β1u1 + β2u2 + β3u3 + β4u4 = (3) 2 4 3 1 2 9 + (0) 6 3 0 2 1 4 + (5) 5 2 1 1 3 0 + (1) 3 2 5 7 1 3 = 6 12 9 3 6 27 + 0 0 0 0 0 0 + 25 10 5 5 15 0 + 3 2 5 7 1 3 = 22 20 1 1 10 24 .

Notice how we could keep our set of vectors fixed, and use different sets of scalars to construct different vectors. You might build a few new linear combinations of u1,u2,u3,u4 right now. We’ll be right here when you get back. What vectors were you able to create? Do you think you could create the vector

w = 13 15 5 17 2 25

with a “suitable” choice of four scalars? Do you think you could create any possible vector from 6 by choosing the proper scalars? These last two questions are very fundamental, and time spent considering them now will prove beneficial later.

Our next two examples are key ones, and a discussion about decompositions is timely. Have a look at Technique DC before studying the next two examples. 

Example ABLC
Archetype B as a linear combination
In this example we will rewrite Archetype B in the language of vectors, vector equality and linear combinations. In Example VESE we wrote the system of m = 3 equations as the vector equality

7x1 6x2 12x3 5x1 + 5x2 + 7x3 x1 + 4x3 = 33 24 5 .

Now we will bust up the linear expressions on the left, first using vector addition,

7x1 5x1 x1 + 6x2 5x2 0x2 + 12x3 7x3 4x3 = 33 24 5 .

Now we can rewrite each of these n = 3 vectors as a scalar multiple of a fixed vector, where the scalar is one of the unknown variables, converting the left-hand side into a linear combination

x1 7 5 1 +x2 6 5 0 +x3 12 7 4 = 33 24 5 .

We can now interpret the problem of solving the system of equations as determining values for the scalar multiples that make the vector equation true. In the analysis of Archetype B, we were able to determine that it had only one solution. A quick way to see this is to row-reduce the coefficient matrix to the 3 × 3 identity matrix and apply Theorem NMRRI to determine that the coefficient matrix is nonsingular. Then Theorem NMUS tells us that the system of equations has a unique solution. This solution is

x1 = 3 x2 = 5 x3 = 2.

So, in the context of this example, we can express the fact that these values of the variables are a solution by writing the linear combination,

(3) 7 5 1 +(5) 6 5 0 +(2) 12 7 4 = 33 24 5 .

Furthermore, these are the only three scalars that will accomplish this equality, since they come from a unique solution.

Notice how the three vectors in this example are the columns of the coefficient matrix of the system of equations. This is our first hint of the important interplay between the vectors that form the columns of a matrix, and the matrix itself.

With any discussion of Archetype A or Archetype B we should be sure to contrast with the other.

Example AALC
Archetype A as a linear combination
As a vector equality, Archetype A can be written as

x1 x2 + 2x3 2x1 + x2 + x3 x1 + x2 = 1 8 5 .

Now bust up the linear expressions on the left, first using vector addition,

x1 2x1 x1 + x2 x2 x2 + 2x3 x3 0x3 = 1 8 5 .

Rewrite each of these n = 3 vectors as a scalar multiple of a fixed vector, where the scalar is one of the unknown variables, converting the left-hand side into a linear combination

x1 1 2 1 +x2 1 1 1 +x3 2 1 0 = 1 8 5 .

Row-reducing the augmented matrix for Archetype A leads to the conclusion that the system is consistent and has free variables, hence infinitely many solutions. So for example, the two solutions

x1 = 2 x2 = 3 x3 = 1 x1 = 3 x2 = 2 x3 = 0

can be used together to say that,

(2) 1 2 1 +(3) 1 1 1 +(1) 2 1 0 = 1 8 5 = (3) 1 2 1 +(2) 1 1 1 +(0) 2 1 0

Ignore the middle of this equation, and move all the terms to the left-hand side,

(2) 1 2 1 +(3) 1 1 1 +(1) 2 1 0 +(3) 1 2 1 +(2) 1 1 1 +(0) 2 1 0 = 0 0 0 .

Regrouping gives

(1) 1 2 1 +(1) 1 1 1 +(1) 2 1 0 = 0 0 0 .

Notice that these three vectors are the columns of the coefficient matrix for the system of equations in Archetype A. This equality says there is a linear combination of those columns that equals the vector of all zeros. Give it some thought, but this says that

x1 = 1 x2 = 1 x3 = 1

is a nontrivial solution to the homogeneous system of equations with the coefficient matrix for the original system in Archetype A. In particular, this demonstrates that this coefficient matrix is singular.

There’s a lot going on in the last two examples. Come back to them in a while and make some connections with the intervening material. For now, we will summarize and explain some of this behavior with a theorem.

Theorem SLSLC
Solutions to Linear Systems are Linear Combinations
Denote the columns of the m × n matrix A as the vectors A1,A2,A3,,An. Then x is a solution to the linear system of equations SA,b if and only if b equals the linear combination of the columns of A formed with the entries of x,

x1A1 + x2A2 + x3A3 + + xnAn = b

Proof   The proof of this theorem is as much about a change in notation as it is about making logical deductions. Write the system of equations SA,b as

a11x1 + a12x2 + a13x3 + + a1nxn = b1 a21x1 + a22x2 + a23x3 + + a2nxn = b2 a31x1 + a32x2 + a33x3 + + a3nxn = b3 am1x1 + am2x2 + am3x3 + + amnxn = bm.

Notice then that the entry of the coefficient matrix A in row i and column j has two names: aij as the coefficient of xj in equation i of the system and Aj i as the i-th entry of the column vector in column j of the coefficient matrix A. Likewise, entry i of b has two names: bi from the linear system and bi as an entry of a vector. Our theorem is an equivalence (Technique E) so we need to prove both “directions.”

( ) Suppose we have the vector equality between b and the linear combination of the columns of A. Then for 1 i m,

bi = bi  Notation = x1A1 + x2A2 + x3A3 + + xnAn i  Hypothesis = x1A1 i + x2A2 i + x3A3 i + + xnAn i  Definition CVA = x1 A1 i + x2 A2 i + x3 A3 i + + xn An i  Definition CVSM = x1ai1 + x2ai2 + x3ai3 + + xnain  Notation = ai1 x1 + ai2 x2 + ai3 x3 + + ain xn  Property CMCN

This says that the entries of x form a solution to equation i of SA,b for all 1 i m, in other words, x is a solution to SA,b.

( ) Suppose now that x is a solution to the linear system SA,b. Then for all 1 i m,

bi = bi  Notation = ai1 x1 + ai2 x2 + ai3 x3 + + ain xn  Hypothesis = x1ai1 + x2ai2 + x3ai3 + + xnain  Property CMCN = x1 A1 i + x2 A2 i + x3 A3 i + + xn An i  Notation = x1A1 i + x2A2 i + x3A3 i + + xnAn i  Definition CVSM = x1A1 + x2A2 + x3A3 + + xnAn i  Definition CVA

Since the components of b and the linear combination of the columns of A agree for all 1 i m, Definition CVE tells us that the vectors are equal.

In other words, this theorem tells us that solutions to systems of equations are linear combinations of the n column vectors of the coefficient matrix (Aj) which yield the constant vector b. Or said another way, a solution to a system of equations SA,b is an answer to the question “How can I form the vector b as a linear combination of the columns of A?” Look through the archetypes that are systems of equations and examine a few of the advertised solutions. In each case use the solution to form a linear combination of the columns of the coefficient matrix and verify that the result equals the constant vector (see Exercise LC.C21).

Subsection VFSS: Vector Form of Solution Sets

We have written solutions to systems of equations as column vectors. For example Archetype B has the solution x1 = 3,x2 = 5,x3 = 2 which we now write as

x = x1 x2 x3 = 3 5 2 .

Now, we will use column vectors and linear combinations to express all of the solutions to a linear system of equations in a compact and understandable way. First, here’s two examples that will motivate our next theorem. This is a valuable technique, almost the equal of row-reducing a matrix, so be sure you get comfortable with it over the course of this section.

Example VFSAD
Vector form of solutions for Archetype D
Archetype D is a linear system of 3 equations in 4 variables. Row-reducing the augmented matrix yields

10324 0 1 3 0 0 00 0 0

and we see r = 2 nonzero rows. Also, D = 1,2 so the dependent variables are then x1 and x2. F = 3,4,5 so the two free variables are x3 and x4. We will express a generic solution for the system by two slightly different methods, though both arrive at the same conclusion.

First, we will decompose (Technique DC) a solution vector. Rearranging each equation represented in the row-reduced form of the augmented matrix by solving for the dependent variable in each row yields the vector equality,

x1 x2 x3 x4 = 4 3x3 + 2x4 x3 + 3x4 x3 x4  Now we will use the definitions of column vector addition and scalar multiplication to express this vector as a linear combination, = 4 0 0 0 + 3x3 x3 x3 0 + 2x4 3x4 0 x 4  Definition CVA = 4 0 0 0 + x3 3 1 1 0 + x4 2 3 0 1  Definition CVSM

We will develop the same linear combination a bit quicker, using three steps. While the method above is instructive, the method below will be our preferred approach.

Step 1. Write the vector of variables as a fixed vector, plus a linear combination of n r vectors, using the free variables as the scalars.

x = x1 x2 x3 x4 = +x3 +x4

Step 2. Use 0’s and 1’s to ensure equality for the entries of the the vectors with indices in F (corresponding to the free variables).

x = x1 x2 x3 x4 = 0 0 +x3 10 +x4 01

Step 3. For each dependent variable, use the augmented matrix to formulate an equation expressing the dependent variable as a constant plus multiples of the free variables. Convert this equation into entries of the vectors that ensure equality for each dependent variable, one at a time.

x1 = 4 3x3 + 2x4 x = x1 x2 x3 x4 = 4 0 0 + x3 3 1 0 + x4 2 0 1 x2 = 0 1x3 + 3x4 x = x1 x2 x3 x4 = 4 0 0 0 + x3 3 1 1 0 + x4 2 3 0 1

This final form of a typical solution is especially pleasing and useful. For example, we can build solutions quickly by choosing values for our free variables, and then compute a linear combination. Such as

x3 = 2,x4 = 5 x = x1 x2 x3 x4 = 4 0 0 0 + (2) 3 1 1 0 + (5) 2 3 0 1 = 12 17 2 5  or, x3 = 1,x4 = 3 x = x1 x2 x3 x4 = 4 0 0 0 + (1) 3 1 1 0 + (3) 2 3 0 1 = 7 8 1 3 .

You’ll find the second solution listed in the write-up for Archetype D, and you might check the first solution by substituting it back into the original equations.

While this form is useful for quickly creating solutions, its even better because it tells us exactly what every solution looks like. We know the solution set is infinite, which is pretty big, but now we can say that a solution is some multiple of 3 1 1 0 plus a multiple of 2 3 0 1 plus the fixed vector 4 0 0 0 . Period. So it only takes us three vectors to describe the entire infinite solution set, provided we also agree on how to combine the three vectors into a linear combination.

This is such an important and fundamental technique, we’ll do another example.

Example VFS
Vector form of solutions
Consider a linear system of m = 5 equations in n = 7 variables, having the augmented matrix A.

A = 2 1 1221 5 21 1 1 3 1 1 1 2 5 1 2 8 5 11615 3 3 9 3 6 5 2 24 21 1 2 11930

Row-reducing we obtain the matrix

B = 10 2 300 9 15 0 5 4 0 0 8 10 0 0 0 0 106 11 0 0 0 0 0 7 21 0 0 0 0 00 0 0

and we see r = 4 nonzero rows. Also, D = 1,2,5,6 so the dependent variables are then x1,x2,x5, and x6. F = 3,4,7,8 so the n r = 3 free variables are x3,x4 and x7. We will express a generic solution for the system by two different methods: both a decomposition and a construction.

First, we will decompose (Technique DC) a solution vector. Rearranging each equation represented in the row-reduced form of the augmented matrix by solving for the dependent variable in each row yields the vector equality,

x1 x2 x3 x4 x5 x6 x7 = 15 2x3 + 3x4 9x7 10 + 5x3 4x4 + 8x7 x3 x4 11 + 6x7 21 7x7 x7  Now we will use the definitions of column vector addition and scalar multiplication to decompose this generic solution vector as a linear combination, = 15 10 0 0 11 21 0 + 2x3 5x3 x3 0 0 0 0 + 3x4 4x4 0 x 4 0 0 0 + 9x7 8x7 0 0 6x7 7x7 x7  Definition CVA = 15 10 0 0 11 21 0 + x3 2 5 1 0 0 0 0 + x4 3 4 0 1 0 0 0 + x7 9 8 0 0 6 7 1  Definition CVSM

We will now develop the same linear combination a bit quicker, using three steps. While the method above is instructive, the method below will be our preferred approach.

Step 1. Write the vector of variables as a fixed vector, plus a linear combination of n r vectors, using the free variables as the scalars.

x = x1 x2 x3 x4 x5 x6 x7 = +x3 +x4 +x7

Step 2. Use 0’s and 1’s to ensure equality for the entries of the the vectors with indices in F (corresponding to the free variables).

x = x1 x2 x3 x4 x5 x6 x7 = 0 00 +x3 10 0 +x4 01 0 +x7 00 1

Step 3. For each dependent variable, use the augmented matrix to formulate an equation expressing the dependent variable as a constant plus multiples of the free variables. Convert this equation into entries of the vectors that ensure equality for each dependent variable, one at a time.

x1 = 15 2x3 + 3x4 9x7 x = x1 x2 x3 x4 x5 x6 x7 = 15 0 0 0 + x3 2 1 0 0 + x4 3 0 1 0 + x7 9 0 0 1 x2 = 10 + 5x3 4x4 + 8x7 x = x1 x2 x3 x4 x5 x6 x7 = 15 10 0 0 0 + x3 2 5 1 0 0 + x4 3 4 0 1 0 + x7 9 8 0 0 1 x5 = 11 + 6x7 x = x1 x2 x3 x4 x5 x6 x7 = 15 10 0 0 11 0 + x3 2 5 1 0 0 0 + x4 3 4 0 1 0 0 + x7 9 8 0 0 6 1 x6 = 21 7x7 x = x1 x2 x3 x4 x5 x6 x7 = 15 10 0 0 11 21 0 + x3 2 5 1 0 0 0 0 + x4 3 4 0 1 0 0 0 + x7 9 8 0 0 6 7 1

This final form of a typical solution is especially pleasing and useful. For example, we can build solutions quickly by choosing values for our free variables, and then compute a linear combination. For example

x3 = 2,x4 = 4,x7 = 3 x = x1 x2 x3 x4 x5 x6 x7 = 15 10 0 0 11 21 0 + (2) 2 5 1 0 0 0 0 + (4) 3 4 0 1 0 0 0 + (3) 9 8 0 0 6 7 1 = 28 40 2 4 29 42 3

or perhaps,

x3 = 5,x4 = 2,x7 = 1 x = x1 x2 x3 x4 x5 x6 x7 = 15 10 0 0 11 21 0 + (5) 2 5 1 0 0 0 0 + (2) 3 4 0 1 0 0 0 + (1) 9 8 0 0 6 7 1 = 2 15 5 2 17 28 1

or even,

x3 = 0,x4 = 0,x7 = 0 x = x1 x2 x3 x4 x5 x6 x7 = 15 10 0 0 11 21 0 + (0) 2 5 1 0 0 0 0 + (0) 3 4 0 1 0 0 0 + (0) 9 8 0 0 6 7 1 = 15 10 0 0 11 21 0

So we can compactly express all of the solutions to this linear system with just 4 fixed vectors, provided we agree how to combine them in a linear combinations to create solution vectors.

Suppose you were told that the vector w below was a solution to this system of equations. Could you turn the problem around and write w as a linear combination of the four vectors c, u1, u2, u3? (See Exercise LC.M11.)

w = 10075 7 9 37 35 8 c = 15 10 0 0 11 21 0 u1 = 2 5 1 0 0 0 0 u2 = 3 4 0 1 0 0 0 u3 = 9 8 0 0 6 7 1

Did you think a few weeks ago that you could so quickly and easily list all the solutions to a linear system of 5 equations in 7 variables?

We’ll now formalize the last two (important) examples as a theorem.

Theorem VFSLS
Vector Form of Solutions to Linear Systems
Suppose that Ab is the augmented matrix for a consistent linear system SA,b of m equations in n variables. Let B be a row-equivalent m × (n + 1) matrix in reduced row-echelon form. Suppose that B has r nonzero rows, columns without leading 1’s with indices F = f1,f2,f3,,fnr,n + 1, and columns with leading 1’s (pivot columns) having indices D = d1,d2,d3,,dr. Define vectors c, uj, 1 j n r of size n by

ci = 0  if i F B k,n+1  if i Di = dk uj i = 1  if i Fi = fj 0  if i Fifj Bk,fj if i Di = dk .

Then the set of solutions to the system of equations SA,b is

S = c + α1u1 + α2u2 + α3u3 + + αnrunrα1,α2,α3,,αnr

Proof   First, SA,b is equivalent to the linear system of equations that has the matrix B as its augmented matrix (Theorem REMES), so we need only show that S is the solution set for the system with B as its augmented matrix. The conclusion of this theorem is that the solution set is equal to the set S, so we will apply Definition SE.

We begin by showing that every element of S is indeed a solution to the system. Let α1,α2,α3,,αnr be one choice of the scalars used to describe elements of S. So an arbitrary element of S, which we will consider as a proposed solution is

x = c + α1u1 + α2u2 + α3u3 + + αnrunr

When r + 1 m, row of the matrix B is a zero row, so the equation represented by that row is always true, no matter which solution vector we propose. So concentrate on rows representing equations 1 r. We evaluate equation of the system represented by B with the proposed solution vector x and refer to the value of the left-hand side of the equation as β,

β = B1 x1 + B2 x2 + B3 x3 + + Bn xn

Since Bdi = 0 for all 1 i r, except that Bd = 1, we see that β simplifies to

β = xd + Bf1 xf1 + Bf2 xf2 + Bf3 xf3 + + Bfnr xfnr

Notice that for 1 i n r

xfi = cfi + α1 u1 fi + α2 u2 fi + α3 u3 fi + + αi ui fi + + αnr unr fi = 0 + α1(0) + α2(0) + α3(0) + + αi(1) + + αnr(0) = αi

So β simplifies further, and we expand the first term

β = xd + Bf1α1 + Bf2α2 + Bf3α3 + + Bfnrαnr = c + α1u1 + α2u2 + α3u3 + + αnrunr d+ Bf1α1 + Bf2α2 + Bf3α3 + + Bfnrαnr = cd + α1 u1 d + α2 u2 d + α3 u3 d + + αnr unr d+ Bf1α1 + Bf2α2 + Bf3α3 + + Bfnrαnr = B,n+1 + α1(B,f1) + α2(B,f2) + α3(B,f3) + + αnr(B,fnr)+ Bf1α1 + Bf2α2 + Bf3α3 + + Bfnrαnr = B,n+1

So β began as the left-hand side of equation of the system represented by B and we now know it equals B,n+1, the constant term for equation of this system. So the arbitrarily chosen vector from S makes every equation of the system true, and therefore is a solution to the system. So all the elements of S are solutions to the system.

For the second half of the proof, assume that x is a solution vector for the system having B as its augmented matrix. For convenience and clarity, denote the entries of x by xi, in other words, xi = xi. We desire to show that this solution vector is also an element of the set S. Begin with the observation that a solution vector’s entries makes equation of the system true for all 1 m,

B,1x1 + B,2x2 + B,3x3 + + B,nxn = B,n+1

When r, the pivot columns of B have zero entries in row with the exception of column d, which will contain a 1. So for 1 r, equation simplifies to

1xd + B,f1xf1 + B,f2xf2 + B,f3xf3 + + B,fnrxfnr = B,n+1

This allows us to write,

xd = xd = B,n+1 B,f1xf1 B,f2xf2 B,f3xf3 B,fnrxfnr = cd + xf1 u1 d + xf2 u2 d + xf3 u3 d + + xfnr unr d = c + xf1u1 + xf2u2 + xf3u3 + + xfnrunr d

This tells us that the entries of the solution vector x corresponding to dependent variables (indices in D), are equal to those of a vector in the set S. We still need to check the other entries of the solution vector x corresponding to the free variables (indices in F) to see if they are equal to the entries of the same vector in the set S. To this end, suppose i F and i = fj. Then

xi = xi = xfj = 0 + 0xf1 + 0xf2 + 0xf3 + + 0xfj1 + 1xfj + 0xfj+1 + + 0xfnr = ci + xf1 u1 i + xf2 u2 i + xf3 u3 i + + xfj uj i + + xfnr unr i = c + xf1u1 + xf2u2 + + xfnrunr i

So entries of x and c + xf1u1 + xf2u2 + + xfnrunr are equal and therefore by Definition CVE they are equal vectors. Since xf1,xf2,xf3,,xfnr are scalars, this shows us that x qualifies for membership in S. So the set S contains all of the solutions to the system.

Note that both halves of the proof of Theorem VFSLS indicate that αi = xfi. In other words, the arbitrary scalars, αi, in the description of the set S actually have more meaning — they are the values of the free variables xfi, 1 i n r. So we will often exploit this observation in our descriptions of solution sets.

Theorem VFSLS formalizes what happened in the three steps of Example VFSAD. The theorem will be useful in proving other theorems, and it it is useful since it tells us an exact procedure for simply describing an infinite solution set. We could program a computer to implement it, once we have the augmented matrix row-reduced and have checked that the system is consistent. By Knuth’s definition, this completes our conversion of linear equation solving from art into science. Notice that it even applies (but is overkill) in the case of a unique solution. However, as a practical matter, I prefer the three-step process of Example VFSAD when I need to describe an infinite solution set. So let’s practice some more, but with a bigger example.

Example VFSAI
Vector form of solutions for Archetype I
Archetype I is a linear system of m = 4 equations in n = 7 variables. Row-reducing the augmented matrix yields

14002 1 34 0 0 0 1 3 5 2 0 00126 6 1 0 0 0 0 0 0 0 0

and we see r = 3 nonzero rows. The columns with leading 1’s are D = {1,3,4} so the r dependent variables are x1,x3,x4. The columns without leading 1’s are F = {2,5,6,7,8}, so the n r = 4 free variables are x2,x5,x6,x7.

Step 1. Write the vector of variables (x) as a fixed vector (c), plus a linear combination of n r = 4 vectors (u1,u2,u3,u4), using the free variables as the scalars.

x = x1 x2 x3 x4 x5 x6 x7 = +x2 +x5 +x6 +x7

Step 2. For each free variable, use 0’s and 1’s to ensure equality for the corresponding entry of the the vectors. Take note of the pattern of 0’s and 1’s at this stage, because this is the best look you’ll have at it. We’ll state an important theorem in the next section and the proof will essentially rely on this observation.

x = x1 x2 x3 x4 x5 x6 x7 = 0 00 0 +x2 1 0 00 +x5 0 1 00 +x6 0 0 10 +x7 0 0 01

Step 3. For each dependent variable, use the augmented matrix to formulate an equation expressing the dependent variable as a constant plus multiples of the free variables. Convert this equation into entries of the vectors that ensure equality for each dependent variable, one at a time.

x1 = 4 4x2 2x5 1x6 + 3x7 x = x1 x2 x3 x4 x5 x6 x7 = 4 0 0 0 0 + x2 4 1 0 0 0 + x5 2 0 1 0 0 + x6 1 0 0 1 0 + x7 3 0 0 0 1 x3 = 2 + 0x2 x5 + 3x6 5x7 x = x1 x2 x3 x4 x5 x6 x7 = 4 0 2 0 0 0 + x2 4 1 0 0 0 0 + x5 2 0 1 1 0 0 + x6 1 0 3 0 1 0 + x7 3 0 5 0 0 1 x4 = 1 + 0x2 2x5 + 6x6 6x7 x = x1 x2 x3 x4 x5 x6 x7 = 4 0 2 1 0 0 0 + x2 4 1 0 0 0 0 0 + x5 2 0 1 2 1 0 0 + x6 1 0 3 6 0 1 0 + x7 3 0 56 0 0 1

We can now use this final expression to quickly build solutions to the system. You might try to recreate each of the solutions listed in the write-up for Archetype I. (Hint: look at the values of the free variables in each solution, and notice that the vector c has 0’s in these locations.)

Even better, we have a description of the infinite solution set, based on just 5 vectors, which we combine in linear combinations to produce solutions.

Whenever we discuss Archetype I you know that’s your cue to go work through Archetype J by yourself. Remember to take note of the 0/1 pattern at the conclusion of Step 2. Have fun — we won’t go anywhere while you’re away.

This technique is so important, that we’ll do one more example. However, an important distinction will be that this system is homogeneous.

Example VFSAL
Vector form of solutions for Archetype L
Archetype L is presented simply as the 5 × 5 matrix

L = 2124 4 6 5 4 4 6 10 7 7 1013 7 5 6 9 10 4 346 6

We’ll interpret it here as the coefficient matrix of a homogeneous system and reference this matrix as L. So we are solving the homogeneous system SL,0 having m = 5 equations in n = 5 variables. If we built the augmented matrix, we would add a sixth column to L containing all zeros. As we did row operations, this sixth column would remain all zeros. So instead we will row-reduce the coefficient matrix, and mentally remember the missing sixth column of zeros. This row-reduced matrix is

100 1 2 0 0 2 2 0 01 2 1 0 0 0 0 0 0 00 0 0

and we see r = 3 nonzero rows. The columns with leading 1’s are D = {1,2,3} so the r dependent variables are x1,x2,x3. The columns without leading 1’s are F = {4,5}, so the n r = 2 free variables are x4,x5. Notice that if we had included the all-zero vector of constants to form the augmented matrix for the system, then the index 6 would have appeared in the set F, and subsequently would have been ignored when listing the free variables.

Step 1. Write the vector of variables (x) as a fixed vector (c), plus a linear combination of n r = 2 vectors (u1,u2), using the free variables as the scalars.

x = x1 x2 x3 x4 x5 = +x4 +x5

Step 2. For each free variable, use 0’s and 1’s to ensure equality for the corresponding entry of the the vectors. Take note of the pattern of 0’s and 1’s at this stage, even if it is not as illuminating as in other examples.

x = x1 x2 x3 x4 x5 = 0 0 +x4 10 +x5 01

Step 3. For each dependent variable, use the augmented matrix to formulate an equation expressing the dependent variable as a constant plus multiples of the free variables. Don’t forget about the “missing” sixth column being full of zeros. Convert this equation into entries of the vectors that ensure equality for each dependent variable, one at a time.

x1 = 0 1x4 + 2x5 x = x1 x2 x3 x4 x5 = 0 0 0 + x4 1 1 0 + x5 2 0 1 x2 = 0 + 2x4 2x5 x = x1 x2 x3 x4 x5 = 0 0 0 0 + x4 1 2 1 0 + x5 2 2 0 1 x3 = 0 2x4 + 1x5 x = x1 x2 x3 x4 x5 = 0 0 0 0 0 + x4 1 2 2 1 0 + x5 2 2 1 0 1

The vector c will always have 0’s in the entries corresponding to free variables. However, since we are solving a homogeneous system, the row-reduced augmented matrix has zeros in column n + 1 = 6, and hence all the entries of c are zero. So we can write

x = x1 x2 x3 x4 x5 = 0+x4 1 2 2 1 0 +x5 2 2 1 0 1 = x4 1 2 2 1 0 +x5 2 2 1 0 1

It will always happen that the solutions to a homogeneous system has c = 0 (even in the case of a unique solution?). So our expression for the solutions is a bit more pleasing. In this example it says that the solutions are all possible linear combinations of the two vectors u1 = 1 2 2 1 0 and u2 = 2 2 1 0 1 , with no mention of any fixed vector entering into the linear combination.

This observation will motivate our next section and the main definition of that section, and after that we will conclude the section by formalizing this situation.

Subsection PSHS: Particular Solutions, Homogeneous Solutions

The next theorem tells us that in order to find all of the solutions to a linear system of equations, it is sufficient to find just one solution, and then find all of the solutions to the corresponding homogeneous system. This explains part of our interest in the null space, the set of all solutions to a homogeneous system.

Theorem PSPHS
Particular Solution Plus Homogeneous Solutions
Suppose that w is one solution to the linear system of equations SA,b. Then y is a solution to SA,b if and only if y = w + z for some vector z NA.

Proof   Let A1,A2,A3,,An be the columns of the coefficient matrix A.

( ) Suppose y = w + z and z NA. Then

b = w1A1 + w2A2 + w3A3 + + wnAn  Theorem SLSLC = w1A1 + w2A2 + w3A3 + + wnAn + 0  Property ZC = w1A1 + w2A2 + w3A3 + + wnAn  Theorem SLSLC + z1A1 + z2A2 + z3A3 + + znAn = w1 + z1 A1 + w2 + z2 A2 + + wn + zn An  Theorem VSPCV = w + z1A1 + w + z2A2 + w + z3A3 + + w + znAn Definition CVA = y1A1 + y2A2 + y3A3 + + ynAn  Definition of y

Applying Theorem SLSLC we see that the vector y is a solution to SA,b.

( ) Suppose y is a solution to SA,b. Then

0 = b b   = y1A1 + y2A2 + y3A3 + + ynAn  Theorem SLSLC w1A1 + w2A2 + w3A3 + + wnAn = y1 w1 A1 + y2 w2 A2 + + yn wn An  Theorem VSPCV = y w1A1 + y w2A2 + y w3A3 + + y wnAn Definition CVA

By Theorem SLSLC we see that the vector y w is a solution to the homogeneous system SA,0 and by Definition NSM, y w NA. In other words, y w = z for some vector z NA. Rewritten, this is y = w + z, as desired.

After proving Theorem NMUS we commented (insufficiently) on the negation of one half of the theorem. Nonsingular coefficient matrices lead to unique solutions for every choice of the vector of constants. What does this say about singular matrices? A singular matrix A has a nontrivial null space (Theorem NMTNS). For a given vector of constants, b, the system SA,b could be inconsistent, meaning there are no solutions. But if there is at least one solution (w), then Theorem PSPHS tells us there will be infinitely many solutions because of the role of the infinite null space for a singular matrix. So a system of equations with a singular coefficient matrix never has a unique solution. Either there are no solutions, or infinitely many solutions, depending on the choice of the vector of constants (b).

Example PSHS
Particular solutions, homogeneous solutions, Archetype D
Archetype D is a consistent system of equations with a nontrivial null space. Let A denote the coefficient matrix of this system. The write-up for this system begins with three solutions,

y1 = 0 1 2 1 y2 = 4 0 0 0 y3 = 7 8 1 3

We will choose to have y1 play the role of w in the statement of Theorem PSPHS, any one of the three vectors listed here (or others) could have been chosen. To illustrate the theorem, we should be able to write each of these three solutions as the vector w plus a solution to the corresponding homogeneous system of equations. Since 0 is always a solution to a homogeneous system we can easily write

y1 = w = w + 0.

The vectors y2 and y3 will require a bit more effort. Solutions to the homogeneous system SA,0 are exactly the elements of the null space of the coefficient matrix, which by an application of Theorem VFSLS is

NA = x3 3 1 1 0 + x4 2 3 0 1 x3,x4

Then

y2 = 4 0 0 0 = 0 1 2 1 + 4 1 21 = 0 1 2 1 +(2) 3 1 1 0 + (1) 2 3 0 1 = w+z2

where

z2 = 4 1 21 = (2) 3 1 1 0 +(1) 2 3 0 1

is obviously a solution of the homogeneous system since it is written as a linear combination of the vectors describing the null space of the coefficient matrix (or as a check, you could just evaluate the equations in the homogeneous system with z2).

Again

y3 = 7 8 1 3 = 0 1 2 1 + 7 7 1 2 = 0 1 2 1 +(1) 3 1 1 0 + 2 2 3 0 1 = w+z3

where

z3 = 7 7 1 2 = (1) 3 1 1 0 +2 2 3 0 1

is obviously a solution of the homogeneous system since it is written as a linear combination of the vectors describing the null space of the coefficient matrix (or as a check, you could just evaluate the equations in the homogeneous system with z2).

Here’s another view of this theorem, in the context of this example. Grab two new solutions of the original system of equations, say

y4 = 11 0 31 y5 = 4 2 4 2

and form their difference,

u = 11 0 31 4 2 4 2 = 152 73 .

It is no accident that u is a solution to the homogeneous system (check this!). In other words, the difference between any two solutions to a linear system of equations is an element of the null space of the coefficient matrix. This is an equivalent way to state Theorem PSPHS. (See Exercise MM.T50).

The ideas of this subsection will be appear again in Chapter LT when we discuss pre-images of linear transformations (Definition PI).

Subsection READ: Reading Questions

  1. Earlier, a reading question asked you to solve the system of equations 2x1 + 3x2 x3 = 0 x1 + 2x2 + x3 = 3 x1 + 3x2 + 3x3 = 7

    Use a linear combination to rewrite this system of equations as a vector equality.

  2. Find a linear combination of the vectors
    S = 1 3 1 , 2 0 4 , 1 3 5

    that equals the vector 1 9 11 .

  3. The matrix below is the augmented matrix of a system of equations, row-reduced to reduced row-echelon form. Write the vector form of the solutions to the system.
    130 6 0 9 0 0 2 0 8 0 00 0 1 3

Subsection EXC: Exercises

C21 Consider each archetype that is a system of equations. For individual solutions listed (both for the original system and the corresponding homogeneous system) express the vector of constants as a linear combination of the columns of the coefficient matrix, as guaranteed by Theorem SLSLC. Verify this equality by computing the linear combination. For systems with no solutions, recognize that it is then impossible to write the vector of constants as a linear combination of the columns of the coefficient matrix. Note too, for homogeneous systems, that the solutions give rise to linear combinations that equal the zero vector.
Archetype A
Archetype B
Archetype C
Archetype D
Archetype E
Archetype F
Archetype G
Archetype H
Archetype I
Archetype J

 
Contributed by Robert Beezer Solution [338]

C22 Consider each archetype that is a system of equations. Write elements of the solution set in vector form, as guaranteed by Theorem VFSLS.
Archetype A
Archetype B
Archetype C
Archetype D
Archetype E
Archetype F
Archetype G
Archetype H
Archetype I
Archetype J

 
Contributed by Robert Beezer Solution [338]

C40 Find the vector form of the solutions to the system of equations below.

2x1 4x2 + 3x3 + x5 = 6 x1 2x2 2x3 + 14x4 4x5 = 15 x1 2x2 + x3 + 2x4 + x5 = 1 2x1 + 4x2 12x4 + x5 = 7

 
Contributed by Robert Beezer Solution [338]

C41 Find the vector form of the solutions to the system of equations below.

2x1 1x2 8x3 + 8x4 + 4x5 9x6 1x7 1x8 18x9 = 3 3x1 2x2 + 5x3 + 2x4 2x5 5x6 + 1x7 + 2x8 + 15x9 = 10 4x1 2x2 + 8x3 + 2x5 14x6 2x8 + 2x9 = 36 1x1 + 2x2 + 1x3 6x4 + 7x6 1x7 3x9 = 8 3x1 + 2x2 + 13x3 14x4 1x5 + 5x6 1x8 + 12x9 = 15 2x1 + 2x2 2x3 4x4 + 1x5 + 6x6 2x7 2x8 15x9 = 7

 
Contributed by Robert Beezer Solution [339]

M10 Example TLC asks if the vector

w = 13 15 5 17 2 25

can be written as a linear combination of the four vectors

u1 = 2 4 3 1 2 9 u2 = 6 3 0 2 1 4 u3 = 5 2 1 1 3 0 u4 = 3 2 5 7 1 3

Can it? Can any vector in 6 be written as a linear combination of the four vectors u1,u2,u3,u4?  
Contributed by Robert Beezer Solution [340]

M11 At the end of Example VFS, the vector w is claimed to be a solution to the linear system under discussion. Verify that w really is a solution. Then determine the four scalars that express w as a linear combination of c, u1, u2, u3.  
Contributed by Robert Beezer Solution [340]

Subsection SOL: Solutions

C21 Contributed by Robert Beezer Statement [334]
Solutions for Archetype A and Archetype B are described carefully in Example AALC and Example ABLC.

C22 Contributed by Robert Beezer Statement [334]
Solutions for Archetype D and Archetype I are described carefully in Example VFSAD and Example VFSAI. The technique described in these examples is probably more useful than carefully deciphering the notation of Theorem VFSLS. The solution for each archetype is contained in its description. So now you can check-off the box for that item.

C40 Contributed by Robert Beezer Statement [335]
Row-reduce the augmented matrix representing this system, to find

120 6 0 1 0 0 4 0 3 0 0 0 0 15 0 0 0 0 0 0

The system is consistent (no leading one in column 6, Theorem RCLS). x2 and x4 are the free variables. Now apply Theorem VFSLS directly, or follow the three-step process of Example VFS, Example VFSAD, Example VFSAI, or Example VFSAL to obtain

x1 x2 x3 x4 x5 = 1 0 3 0 5 +x2 2 1 0 0 0 +x4 6 0 4 1 0

C41 Contributed by Robert Beezer Statement [335]
Row-reduce the augmented matrix representing this system, to find

10320100 3 6 0 2 4 0 3 0 0 2 1 0 00 0 12001 3 0 0 0 0 0 0 0 4 0 0 00 0 0 0 01 2 2 0 0 0 0 0 0 0 0 0 0

The system is consistent (no leading one in column 10, Theorem RCLS). F = 3,4,6,9,10, so the free variables are x3,x4,x6 and x9. Now apply Theorem VFSLS directly, or follow the three-step process of Example VFS, Example VFSAD, Example VFSAI, or Example VFSAL to obtain the solution set

S = 6 1 0 0 3 0 0 2 0 + x3 3 2 1 0 0 0 0 0 0 + x4 2 4 0 1 0 0 0 0 0 + x6 1 3 0 0 2 1 0 0 0 + x9 3 2 0 0 1 0 4 2 1 x3,x4,x6,x9

M10 Contributed by Robert Beezer Statement [336]
No, it is not possible to create w as a linear combination of the four vectors u1,u2,u3,u4. By creating the desired linear combination with unknowns as scalars, Theorem SLSLC provides a system of equations that has no solution. This one computation is enough to show us that it is not possible to create all the vectors of 6 through linear combinations of the four vectors u1,u2,u3,u4.

M11 Contributed by Robert Beezer Statement [337]
The coefficient of c is 1. The coefficients of u1, u2, u3 lie in the third, fourth and seventh entries of w. Can you see why? (Hint: F = 3,4,7,8, so the free variables are x3,x4 and x7.)