Skip to main content
\(\newcommand{\orderof}[1]{\sim #1} \newcommand{\Z}{\mathbb{Z}} \newcommand{\reals}{\mathbb{R}} \newcommand{\real}[1]{\mathbb{R}^{#1}} \newcommand{\complexes}{\mathbb{C}} \newcommand{\complex}[1]{\mathbb{C}^{#1}} \newcommand{\conjugate}[1]{\overline{#1}} \newcommand{\modulus}[1]{\left\lvert#1\right\rvert} \newcommand{\zerovector}{\vect{0}} \newcommand{\zeromatrix}{\mathcal{O}} \newcommand{\innerproduct}[2]{\left\langle#1,\,#2\right\rangle} \newcommand{\norm}[1]{\left\lVert#1\right\rVert} \newcommand{\dimension}[1]{\dim\left(#1\right)} \newcommand{\nullity}[1]{n\left(#1\right)} \newcommand{\rank}[1]{r\left(#1\right)} \newcommand{\ds}{\oplus} \newcommand{\detname}[1]{\det\left(#1\right)} \newcommand{\detbars}[1]{\left\lvert#1\right\rvert} \newcommand{\trace}[1]{t\left(#1\right)} \newcommand{\sr}[1]{#1^{1/2}} \newcommand{\spn}[1]{\left\langle#1\right\rangle} \newcommand{\nsp}[1]{\mathcal{N}\!\left(#1\right)} \newcommand{\csp}[1]{\mathcal{C}\!\left(#1\right)} \newcommand{\rsp}[1]{\mathcal{R}\!\left(#1\right)} \newcommand{\lns}[1]{\mathcal{L}\!\left(#1\right)} \newcommand{\per}[1]{#1^\perp} \newcommand{\augmented}[2]{\left\lbrack\left.#1\,\right\rvert\,#2\right\rbrack} \newcommand{\linearsystem}[2]{\mathcal{LS}\!\left(#1,\,#2\right)} \newcommand{\homosystem}[1]{\linearsystem{#1}{\zerovector}} \newcommand{\rowopswap}[2]{R_{#1}\leftrightarrow R_{#2}} \newcommand{\rowopmult}[2]{#1R_{#2}} \newcommand{\rowopadd}[3]{#1R_{#2}+R_{#3}} \newcommand{\leading}[1]{\boxed{#1}} \newcommand{\rref}{\xrightarrow{\text{RREF}}} \newcommand{\elemswap}[2]{E_{#1,#2}} \newcommand{\elemmult}[2]{E_{#2}\left(#1\right)} \newcommand{\elemadd}[3]{E_{#2,#3}\left(#1\right)} \newcommand{\scalarlist}[2]{{#1}_{1},\,{#1}_{2},\,{#1}_{3},\,\ldots,\,{#1}_{#2}} \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\colvector}[1]{\begin{bmatrix}#1\end{bmatrix}} \newcommand{\vectorcomponents}[2]{\colvector{#1_{1}\\#1_{2}\\#1_{3}\\\vdots\\#1_{#2}}} \newcommand{\vectorlist}[2]{\vect{#1}_{1},\,\vect{#1}_{2},\,\vect{#1}_{3},\,\ldots,\,\vect{#1}_{#2}} \newcommand{\vectorentry}[2]{\left\lbrack#1\right\rbrack_{#2}} \newcommand{\matrixentry}[2]{\left\lbrack#1\right\rbrack_{#2}} \newcommand{\lincombo}[3]{#1_{1}\vect{#2}_{1}+#1_{2}\vect{#2}_{2}+#1_{3}\vect{#2}_{3}+\cdots +#1_{#3}\vect{#2}_{#3}} \newcommand{\matrixcolumns}[2]{\left\lbrack\vect{#1}_{1}|\vect{#1}_{2}|\vect{#1}_{3}|\ldots|\vect{#1}_{#2}\right\rbrack} \newcommand{\transpose}[1]{#1^{t}} \newcommand{\inverse}[1]{#1^{-1}} \newcommand{\submatrix}[3]{#1\left(#2|#3\right)} \newcommand{\adj}[1]{\transpose{\left(\conjugate{#1}\right)}} \newcommand{\adjoint}[1]{#1^\ast} \newcommand{\set}[1]{\left\{#1\right\}} \newcommand{\setparts}[2]{\left\lbrace#1\,\middle|\,#2\right\rbrace} \newcommand{\card}[1]{\left\lvert#1\right\rvert} \newcommand{\setcomplement}[1]{\overline{#1}} \newcommand{\charpoly}[2]{p_{#1}\left(#2\right)} \newcommand{\eigenspace}[2]{\mathcal{E}_{#1}\left(#2\right)} \newcommand{\eigensystem}[3]{\lambda&=#2&\eigenspace{#1}{#2}&=\spn{\set{#3}}} \newcommand{\geneigenspace}[2]{\mathcal{G}_{#1}\left(#2\right)} \newcommand{\algmult}[2]{\alpha_{#1}\left(#2\right)} \newcommand{\geomult}[2]{\gamma_{#1}\left(#2\right)} \newcommand{\indx}[2]{\iota_{#1}\left(#2\right)} \newcommand{\ltdefn}[3]{#1\colon #2\rightarrow#3} \newcommand{\lteval}[2]{#1\left(#2\right)} \newcommand{\ltinverse}[1]{#1^{-1}} \newcommand{\restrict}[2]{{#1}|_{#2}} \newcommand{\preimage}[2]{#1^{-1}\left(#2\right)} \newcommand{\rng}[1]{\mathcal{R}\!\left(#1\right)} \newcommand{\krn}[1]{\mathcal{K}\!\left(#1\right)} \newcommand{\compose}[2]{{#1}\circ{#2}} \newcommand{\vslt}[2]{\mathcal{LT}\left(#1,\,#2\right)} \newcommand{\isomorphic}{\cong} \newcommand{\similar}[2]{\inverse{#2}#1#2} \newcommand{\vectrepname}[1]{\rho_{#1}} \newcommand{\vectrep}[2]{\lteval{\vectrepname{#1}}{#2}} \newcommand{\vectrepinvname}[1]{\ltinverse{\vectrepname{#1}}} \newcommand{\vectrepinv}[2]{\lteval{\ltinverse{\vectrepname{#1}}}{#2}} \newcommand{\matrixrep}[3]{M^{#1}_{#2,#3}} \newcommand{\matrixrepcolumns}[4]{\left\lbrack \left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{1}}}\right|\left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{2}}}\right|\left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{3}}}\right|\ldots\left|\vectrep{#2}{\lteval{#1}{\vect{#3}_{#4}}}\right.\right\rbrack} \newcommand{\cbm}[2]{C_{#1,#2}} \newcommand{\jordan}[2]{J_{#1}\left(#2\right)} \newcommand{\hadamard}[2]{#1\circ #2} \newcommand{\hadamardidentity}[1]{J_{#1}} \newcommand{\hadamardinverse}[1]{\widehat{#1}} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \)

SectionFSFour Subsets

There are four natural subsets associated with a matrix. We have met three already: the null space, the column space and the row space. In this section we will introduce a fourth, the left null space. The objective of this section is to describe one procedure that will allow us to find linearly independent sets that span each of these four sets of column vectors. Along the way, we will make a connection with the inverse of a matrix, so Theorem FS will tie together most all of this chapter (and the entire course so far).

SubsectionLNSLeft Null Space

DefinitionLNSLeft Null Space

Suppose \(A\) is an \(m\times n\) matrix. Then the left null space is defined as \(\lns{A}=\nsp{\transpose{A}}\subseteq\complex{m}\text{.}\)

The left null space will not feature prominently in the sequel, but we can explain its name and connect it to row operations. Suppose \(\vect{y}\in\lns{A}\text{.}\) Then by Definition LNS, \(\transpose{A}\vect{y}=\zerovector\text{.}\) We can then write \begin{align*} \transpose{\zerovector} &=\transpose{\left(\transpose{A}\vect{y}\right)}&& \knowl{./knowl/definition-LNS.html}{\text{Definition LNS}}\\ &=\transpose{\vect{y}}\transpose{\left(\transpose{A}\right)}&& \knowl{./knowl/theorem-MMT.html}{\text{Theorem MMT}}\\ &=\transpose{\vect{y}}A&& \knowl{./knowl/theorem-TT.html}{\text{Theorem TT}}\text{.} \end{align*}

The product \(\transpose{\vect{y}}A\) can be viewed as the components of \(\vect{y}\) acting as the scalars in a linear combination of the rows of \(A\text{.}\) And the result is a row vector, \(\transpose{\zerovector}\) that is totally zeros. When we apply a sequence of row operations to a matrix, each row of the resulting matrix is some linear combination of the rows. These observations tell us that the vectors in the left null space are scalars that record a sequence of row operations that result in a row of zeros in the row-reduced version of the matrix. We will see this idea more explicitly in the course of proving Theorem FS.

SubsectionCCSComputing Column Spaces

We have three ways to build the column space of a matrix. First, we can use just the definition, Definition CSM, and express the column space as a span of the columns of the matrix. A second approach gives us the column space as the span of some of the columns of the matrix, and additionally, this set is linearly independent (Theorem BCS). Finally, we can transpose the matrix, row-reduce the transpose, kick out zero rows, and write the remaining rows as column vectors. Theorem CSRST and Theorem BRS tell us that the resulting vectors are linearly independent and their span is the column space of the original matrix.

We will now demonstrate a fourth method by way of a rather complicated example. Study this example carefully, but realize that its main purpose is to motivate a theorem that simplifies much of the apparent complexity. So other than an instructive exercise or two, the procedure we are about to describe will not be a usual approach to computing a column space.

This example motivates the remainder of this section, so it is worth careful study. You might attempt to mimic the second approach with the coefficient matrices of Archetype I and Archetype J. We will see shortly that the matrix \(L\) contains more information about \(A\) than just the column space.

SubsectionEEFExtended Echelon Form

The final matrix that we row-reduced in Example CSANS should look familiar in most respects to the procedure we used to compute the inverse of a nonsingular matrix, Theorem CINM. We will now generalize that procedure to matrices that are not necessarily nonsingular, or even square. First a definition.

DefinitionEEFExtended Echelon Form

Suppose \(A\) is an \(m\times n\) matrix. Extend \(A\) on its right side with the addition of an \(m\times m\) identity matrix to form an \(m\times (n + m)\) matrix \(M\text{.}\) Use row operations to bring \(M\) to reduced row-echelon form and call the result \(N\text{.}\) \(N\) is the extended reduced row-echelon form of \(A\text{,}\) and we will standardize on names for five submatrices (\(B\text{,}\) \(C\text{,}\) \(J\text{,}\) \(K\text{,}\) \(L\)) of \(N\text{.}\)

Let \(B\) denote the \(m\times n\) matrix formed from the first \(n\) columns of \(N\) and let \(J\) denote the \(m\times m\) matrix formed from the last \(m\) columns of \(N\text{.}\) Suppose that \(B\) has \(r\) nonzero rows. Further partition \(N\) by letting \(C\) denote the \(r\times n\) matrix formed from all of the nonzero rows of \(B\text{.}\) Let \(K\) be the \(r\times m\) matrix formed from the first \(r\) rows of \(J\text{,}\) while \(L\) will be the \((m-r)\times m\) matrix formed from the bottom \(m-r\) rows of \(J\text{.}\) Pictorially, \begin{equation*} M=[A\vert I_m] \rref N=[B\vert J] = \left[\begin{array}{c|c}C&K\\\hline0&L\end{array}\right]\text{.} \end{equation*}

Proof

Notice that in the case where \(A\) is a nonsingular matrix we know that the reduced row-echelon form of \(A\) is the identity matrix (Theorem NMRRI), so \(B=I_n\text{.}\) Then the second conclusion above says \(JA=B=I_n\text{,}\) so \(J\) is the inverse of \(A\text{.}\) Thus this theorem generalizes Theorem CINM, though the result is a “left-inverse” of \(A\) rather than a “right-inverse.”

The third conclusion of Theorem PEEF is the most telling. It says that \(\vect{x}\) is a solution to the linear system \(\linearsystem{A}{\vect{y}}\) if and only if \(\vect{x}\) is a solution to the linear system \(\linearsystem{B}{J\vect{y}}\text{.}\) Or said differently, if we row-reduce the augmented matrix \(\augmented{A}{\vect{y}}\) we will get the augmented matrix \(\augmented{B}{J\vect{y}}\text{.}\) The matrix \(J\) tracks the cumulative effect of the row operations that converts \(A\) to reduced row-echelon form, here effectively applying them to the vector of constants in a system of equations having \(A\) as a coefficient matrix. When \(A\) row-reduces to a matrix with zero rows, then \(J\vect{y}\) should also have zero entries in the same rows if the system is to be consistent.

SubsectionFSFour Subsets

With all the preliminaries in place we can state our main result for this section. In essence this result will allow us to say that we can find linearly independent sets to use in span constructions for all four subsets (null space, column space, row space, left null space) by analyzing only the extended echelon form of the matrix, and specifically, just the two submatrices \(C\) and \(L\text{,}\) which will be ripe for analysis since they are already in reduced row-echelon form (Theorem PEEF).

Proof

The first two conclusions of this theorem are nearly trivial. But they set up a pattern of results for \(C\) that is reflected in the latter two conclusions about \(L\text{.}\) In total, they tell us that we can compute all four subsets just by finding null spaces and row spaces. This theorem does not tell us exactly how to compute these subsets, but instead simply expresses them as null spaces and row spaces of matrices in reduced row-echelon form without any zero rows (\(C\) and \(L\)). A linearly independent set that spans the null space of a matrix in reduced row-echelon form can be found easily with Theorem BNS. It is an even easier matter to find a linearly independent set that spans the row space of a matrix in reduced row-echelon form with Theorem BRS, especially when there are no zero rows present. So an application of Theorem FS is typically followed by two applications each of Theorem BNS and Theorem BRS.

The situation when \(r=m\) deserves comment, since now the matrix \(L\) has no rows. What is \(\csp{A}\) when we try to apply Theorem FS and encounter \(\nsp{L}\text{?}\) One interpretation of this situation is that \(L\) is the coefficient matrix of a homogeneous system that has no equations. How hard is it to find a solution vector to this system? Some thought will convince you that any proposed vector will qualify as a solution, since it makes all of the equations true. So every possible vector is in the null space of \(L\) and therefore \(\csp{A}=\nsp{L}=\complex{m}\text{.}\) OK, perhaps this sounds like some twisted argument from Alice in Wonderland. Let us try another argument that might solidly convince you of this logic.

If \(r=m\text{,}\) when we row-reduce the augmented matrix of \(\linearsystem{A}{\vect{b}}\) the result will have no zero rows, and the first \(n\) columns will all be pivot columns, leaving none for the final column, so by Theorem RCLS the system will be consistent. By Theorem CSCS, \(\vect{b}\in\csp{A}\text{.}\) Since \(\vect{b}\) was arbitrary, every possible vector is in the column space of \(A\text{,}\) so we again have \(\csp{A}=\complex{m}\text{.}\) The situation when a matrix has \(r=m\) is known by the term full rank, and in the case of a square matrix coincides with nonsingularity (see Exercise FS.M50).

The properties of the matrix \(L\) described by this theorem can be explained informally as follows. A column vector \(\vect{y}\in\complex{m}\) is in the column space of \(A\) if the linear system \(\linearsystem{A}{\vect{y}}\) is consistent (Theorem CSCS). By Theorem RCLS, the reduced row-echelon form of the augmented matrix \(\augmented{A}{\vect{y}}\) of a consistent system will have zeros in the bottom \(m-r\) locations of the last column. By Theorem PEEF this final column is the vector \(J\vect{y}\) and so should then have zeros in the final \(m-r\) locations. But since \(L\) comprises the final \(m-r\) rows of \(J\text{,}\) this condition is expressed by saying \(\vect{y}\in\nsp{L}\text{.}\)

Additionally, the rows of \(J\) are the scalars in linear combinations of the rows of \(A\) that create the rows of \(B\text{.}\) That is, the rows of \(J\) record the net effect of the sequence of row operations that takes \(A\) to its reduced row-echelon form, \(B\text{.}\) This can be seen in the equation \(JA=B\) (Theorem PEEF). As such, the rows of \(L\) are scalars for linear combinations of the rows of \(A\) that yield zero rows. But such linear combinations are precisely the elements of the left null space. So any element of the row space of \(L\) is also an element of the left null space of \(A\text{.}\)

We will now illustrate Theorem FS with a few examples.

The next example is just a bit different since the matrix has more rows than columns, and a trivial null space.

Example COV and Example CSROI each describes the column space of the coefficient matrix from Archetype I as the span of a set of \(r=3\) linearly independent vectors. It is no accident that these two different sets both have the same size. If we (you?) were to calculate the column space of this matrix using the null space of the matrix \(L\) from Theorem FS then we would again find a set of 3 linearly independent vectors that span the range. More on this later.

So we have three different methods to obtain a description of the column space of a matrix as the span of a linearly independent set. Theorem BCS is sometimes useful since the vectors it specifies are equal to actual columns of the matrix. Theorem BRS and Theorem CSRST combine to create vectors with lots of zeros, and strategically placed 1's near the top of the vector. Theorem FS and the matrix \(L\) from the extended echelon form gives us a third method, which tends to create vectors with lots of zeros, and strategically placed 1's near the bottom of the vector. If we do not care about linear independence we can also appeal to Definition CSM and simply express the column space as the span of all the columns of the matrix, giving us a fourth description.

With Theorem CSRST and Definition RSM, we can compute column spaces with theorems about row spaces, and we can compute row spaces with theorems about column spaces, but in each case we must transpose the matrix first. At this point you may be overwhelmed by all the possibilities for computing column and row spaces. Figure 3.120 is meant to help. For both the column space and row space, it suggests four techniques. One is to appeal to the definition, another yields a span of a linearly independent set, and a third uses Theorem FS. A fourth suggests transposing the matrix and the dashed line implies that then the companion set of techniques can be applied. This can lead to a bit of silliness, since if you were to follow the dashed lines twice you would transpose the matrix twice, and by Theorem TT would accomplish nothing productive.

<<SVG image is unavailable, or your browser cannot render it>>

Figure3.120Column Space and Row Space Techniques

Although we have many ways to describe a column space, notice that one tempting strategy will usually fail. It is not possible to simply row-reduce a matrix directly and then use the columns of the row-reduced matrix as a set whose span equals the column space. In other words, row operations do not preserve column spaces (however row operations do preserve row spaces, Theorem REMRS). See Exercise CRS.M21.

SubsectionReading Questions

1

Find a nontrivial element of the left null space of \(A\text{.}\) \begin{equation*} A= \begin{bmatrix} 2 & 1 & -3 & 4\\ -1 & -1 & 2 & -1\\ 0 & -1 & 1 & 2 \end{bmatrix} \end{equation*}

2

Find the matrices \(C\) and \(L\) in the extended echelon form of \(A\text{.}\) \begin{equation*} A= \begin{bmatrix} -9 & 5 & -3 \\ 2 & -1 & 1 \\ -5 & 3 & -1 \end{bmatrix} \end{equation*}

SubsectionExercises

C20

Example FSAG concludes with several questions. Perform the analysis suggested by these questions.

C25

Given the matrix \(A\) below, use the extended echelon form of \(A\) to answer each part of this problem. In each part, find a linearly independent set of vectors, \(S\text{,}\) so that the span of \(S\text{,}\) \(\spn{S}\text{,}\) equals the specified set of vectors.

  1. The row space of \(A\text{,}\) \(\rsp{A}\text{.}\)
  2. The column space of \(A\text{,}\) \(\csp{A}\text{.}\)
  3. The null space of \(A\text{,}\) \(\nsp{A}\text{.}\)
  4. The left null space of \(A\text{,}\) \(\lns{A}\text{.}\)

\begin{equation*} A= \begin{bmatrix} -5 & 3 & -1 \\ -1 & 1 & 1 \\ -8 & 5 & -1 \\ 3 & -2 & 0 \end{bmatrix} \end{equation*}

Solution
C26

For the matrix \(D\) below use the extended echelon form to find:

  1. A linearly independent set whose span is the column space of \(D\text{.}\)
  2. A linearly independent set whose span is the left null space of \(D\text{.}\)

\begin{align*} D&= \begin{bmatrix} -7 & -11 & -19 & -15\\ 6 & 10 & 18 & 14 \\ 3 & 5 & 9 & 7 \\ -1 & -2 & -4 & -3 \end{bmatrix} \end{align*}

Solution
C41

The following archetypes are systems of equations. For each system, write the vector of constants as a linear combination of the vectors in the span construction for the column space provided by Theorem FS and Theorem BNS (these vectors are listed for each of these archetypes).

Archetype A, Archetype B, Archetype C, Archetype D, Archetype E, Archetype F, Archetype G, Archetype H, Archetype I, Archetype J

C43

The following archetypes are either matrices or systems of equations with coefficient matrices. For each matrix, compute the extended echelon form \(N\) and identify the matrices \(C\) and \(L\text{.}\) Using Theorem FS, Theorem BNS and Theorem BRS express the null space, the row space, the column space and left null space of each coefficient matrix as a span of a linearly independent set.

Archetype A, Archetype B, Archetype C, Archetype D/Archetype E, Archetype F, Archetype G/Archetype H, Archetype I, Archetype J, Archetype K, Archetype L

C60

For the matrix \(B\) below, find sets of vectors whose span equals the column space of \(B\) (\(\csp{B}\)) and which individually meet the following extra requirements.

  1. The set illustrates the definition of the column space.
  2. The set is linearly independent and the members of the set are columns of \(B\text{.}\)
  3. The set is linearly independent with a “nice pattern of zeros and ones” at the top of each vector.
  4. The set is linearly independent with a “nice pattern of zeros and ones” at the bottom of each vector.

\begin{equation*} B= \begin{bmatrix} 2 & 3 & 1 & 1\\ 1 & 1 & 0 & 1\\ -1 & 2 & 3 & -4 \end{bmatrix} \end{equation*}

Solution
C61

Let \(A\) be the matrix below, and find the indicated sets with the requested properties.

  1. A linearly independent set \(S\) so that \(\csp{A}=\spn{S}\) and \(S\) is composed of columns of \(A\text{.}\)
  2. A linearly independent set \(S\) so that \(\csp{A}=\spn{S}\) and the vectors in \(S\) have a nice pattern of zeros and ones at the top of the vectors.
  3. A linearly independent set \(S\) so that \(\csp{A}=\spn{S}\) and the vectors in \(S\) have a nice pattern of zeros and ones at the bottom of the vectors.
  4. A linearly independent set \(S\) so that \(\rsp{A}=\spn{S}\text{.}\)

\begin{equation*} A= \begin{bmatrix} 2 & -1 & 5 & -3\\ -5 & 3 & -12 & 7\\ 1 & 1 & 4 & -3 \end{bmatrix} \end{equation*}

Solution
M50

Suppose that \(A\) is a nonsingular matrix. Extend the four conclusions of Theorem FS in this special case and discuss connections with previous results (such as Theorem NME4).

M51

Suppose that \(A\) is a singular matrix. Extend the four conclusions of Theorem FS in this special case and discuss connections with previous results (such as Theorem NME4).