Skip to main content
\(\newcommand{\orderof}[1]{\sim #1} \newcommand{\Z}{\mathbb{Z}} \newcommand{\reals}{\mathbb{R}} \newcommand{\real}[1]{\mathbb{R}^{#1}} \newcommand{\complexes}{\mathbb{C}} \newcommand{\complex}[1]{\mathbb{C}^{#1}} \newcommand{\conjugate}[1]{\overline{#1}} \newcommand{\modulus}[1]{\left\lvert#1\right\rvert} \newcommand{\zerovector}{\vect{0}} \newcommand{\zeromatrix}{\mathcal{O}} \newcommand{\innerproduct}[2]{\left\langle#1,\,#2\right\rangle} \newcommand{\norm}[1]{\left\lVert#1\right\rVert} \newcommand{\dimension}[1]{\dim\left(#1\right)} \newcommand{\nullity}[1]{n\left(#1\right)} \newcommand{\rank}[1]{r\left(#1\right)} \newcommand{\ds}{\oplus} \newcommand{\detname}[1]{\det\left(#1\right)} \newcommand{\detbars}[1]{\left\lvert#1\right\rvert} \newcommand{\trace}[1]{t\left(#1\right)} \newcommand{\sr}[1]{#1^{1/2}} \newcommand{\spn}[1]{\left\langle#1\right\rangle} \newcommand{\nsp}[1]{\mathcal{N}\!\left(#1\right)} \newcommand{\csp}[1]{\mathcal{C}\!\left(#1\right)} \newcommand{\rsp}[1]{\mathcal{R}\!\left(#1\right)} \newcommand{\lns}[1]{\mathcal{L}\!\left(#1\right)} \newcommand{\per}[1]{#1^\perp} \newcommand{\augmented}[2]{\left\lbrack\left.#1\,\right\rvert\,#2\right\rbrack} \newcommand{\linearsystem}[2]{\mathcal{LS}\!\left(#1,\,#2\right)} \newcommand{\homosystem}[1]{\linearsystem{#1}{\zerovector}} \newcommand{\rowopswap}[2]{R_{#1}\leftrightarrow R_{#2}} \newcommand{\rowopmult}[2]{#1R_{#2}} \newcommand{\rowopadd}[3]{#1R_{#2}+R_{#3}} \newcommand{\leading}[1]{\boxed{#1}} \newcommand{\rref}{\xrightarrow{\text{RREF}}} \newcommand{\elemswap}[2]{E_{#1,#2}} \newcommand{\elemmult}[2]{E_{#2}\left(#1\right)} \newcommand{\elemadd}[3]{E_{#2,#3}\left(#1\right)} \newcommand{\scalarlist}[2]{{#1}_{1},\,{#1}_{2},\,{#1}_{3},\,\ldots,\,{#1}_{#2}} \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\colvector}[1]{\begin{bmatrix}#1\end{bmatrix}} \newcommand{\vectorcomponents}[2]{\colvector{#1_{1}\\#1_{2}\\#1_{3}\\\vdots\\#1_{#2}}} \newcommand{\vectorlist}[2]{\vect{#1}_{1},\,\vect{#1}_{2},\,\vect{#1}_{3},\,\ldots,\,\vect{#1}_{#2}} \newcommand{\vectorentry}[2]{\left\lbrack#1\right\rbrack_{#2}} \newcommand{\matrixentry}[2]{\left\lbrack#1\right\rbrack_{#2}} \newcommand{\lincombo}[3]{#1_{1}\vect{#2}_{1}+#1_{2}\vect{#2}_{2}+#1_{3}\vect{#2}_{3}+\cdots +#1_{#3}\vect{#2}_{#3}} \newcommand{\matrixcolumns}[2]{\left\lbrack\vect{#1}_{1}|\vect{#1}_{2}|\vect{#1}_{3}|\ldots|\vect{#1}_{#2}\right\rbrack} \newcommand{\transpose}[1]{#1^{t}} \newcommand{\inverse}[1]{#1^{-1}} \newcommand{\submatrix}[3]{#1\left(#2|#3\right)} \newcommand{\adj}[1]{\transpose{\left(\conjugate{#1}\right)}} \newcommand{\adjoint}[1]{#1^\ast} \newcommand{\set}[1]{\left\{#1\right\}} \newcommand{\setparts}[2]{\left\lbrace#1\,\middle|\,#2\right\rbrace} \newcommand{\card}[1]{\left\lvert#1\right\rvert} \newcommand{\setcomplement}[1]{\overline{#1}} \newcommand{\charpoly}[2]{p_{#1}\left(#2\right)} \newcommand{\eigenspace}[2]{\mathcal{E}_{#1}\left(#2\right)} \newcommand{\eigensystem}[3]{\lambda&=#2&\eigenspace{#1}{#2}&=\spn{\set{#3}}} \newcommand{\geneigenspace}[2]{\mathcal{G}_{#1}\left(#2\right)} \newcommand{\algmult}[2]{\alpha_{#1}\left(#2\right)} \newcommand{\geomult}[2]{\gamma_{#1}\left(#2\right)} \newcommand{\indx}[2]{\iota_{#1}\left(#2\right)} \newcommand{\ltdefn}[3]{#1\colon #2\rightarrow#3} \newcommand{\lteval}[2]{#1\left(#2\right)} \newcommand{\ltinverse}[1]{#1^{-1}} \newcommand{\restrict}[2]{{#1}|_{#2}} \newcommand{\preimage}[2]{#1^{-1}\left(#2\right)} \newcommand{\rng}[1]{\mathcal{R}\!\left(#1\right)} \newcommand{\krn}[1]{\mathcal{K}\!\left(#1\right)} \newcommand{\compose}[2]{{#1}\circ{#2}} \newcommand{\vslt}[2]{\mathcal{LT}\left(#1,\,#2\right)} \newcommand{\isomorphic}{\cong} \newcommand{\similar}[2]{\inverse{#2}#1#2} \newcommand{\vectrepname}[1]{\rho_{#1}} \newcommand{\vectrep}[2]{\lteval{\vectrepname{#1}}{#2}} \newcommand{\vectrepinvname}[1]{\ltinverse{\vectrepname{#1}}} \newcommand{\vectrepinv}[2]{\lteval{\ltinverse{\vectrepname{#1}}}{#2}} \newcommand{\matrixrep}[3]{M^{#1}_{#2,#3}} \newcommand{\matrixrepcolumns}[4]{\left\lbrack \left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{1}}}\right|\left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{2}}}\right|\left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{3}}}\right|\ldots\left|\vectrep{#2}{\lteval{#1}{\vect{#3}_{#4}}}\right.\right\rbrack} \newcommand{\cbm}[2]{C_{#1,#2}} \newcommand{\jordan}[2]{J_{#1}\left(#2\right)} \newcommand{\hadamard}[2]{#1\circ #2} \newcommand{\hadamardidentity}[1]{J_{#1}} \newcommand{\hadamardinverse}[1]{\widehat{#1}} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \)

SectionBBases

A basis of a vector space is one of the most useful concepts in linear algebra. It often provides a concise, finite description of an infinite vector space.

SubsectionBBases

We now have all the tools in place to define a basis of a vector space.

DefinitionBBasis

Suppose \(V\) is a vector space. Then a subset \(S\subseteq V\) is a basis of \(V\) if it is linearly independent and spans \(V\text{.}\)

So, a basis is a linearly independent spanning set for a vector space. The requirement that the set spans \(V\) insures that \(S\) has enough raw material to build \(V\text{,}\) while the linear independence requirement insures that we do not have any more raw material than we need. As we shall see soon in Section D, a basis is a minimal spanning set.

You may have noticed that we used the term basis for some of the titles of previous theorems (e.g. Theorem BNS, Theorem BCS, Theorem BRS) and if you review each of these theorems you will see that their conclusions provide linearly independent spanning sets for sets that we now recognize as subspaces of \(\complex{m}\text{.}\) Examples associated with these theorems include Example NSLIL, Example CSOCD and Example IAS. As we will see, these three theorems will continue to be powerful tools, even in the setting of more general vector spaces.

Furthermore, the archetypes contain an abundance of bases. For each coefficient matrix of a system of equations, and for each archetype defined simply as a matrix, there is a basis for the null space, three bases for the column space, and a basis for the row space. For this reason, our subsequent examples will concentrate on bases for vector spaces other than \(\complex{m}\text{.}\)

Notice that Definition B does not preclude a vector space from having many bases, and this is the case, as hinted above by the statement that the archetypes contain three bases for the column space of a matrix. More generally, we can grab any basis for a vector space, multiply any one basis vector by a nonzero scalar and create a slightly different set that is still a basis. For “important” vector spaces, it will be convenient to have a collection of “nice” bases. When a vector space has a single particularly nice basis, it is sometimes called the standard basis though there is nothing precise enough about this term to allow us to define it formally — it is a question of style. Here are some nice bases for important vector spaces.

Proof

The bases described above will often be convenient ones to work with. However a basis does not have to obviously look like a basis.

We have seen that several of the sets associated with a matrix are subspaces of vector spaces of column vectors. Specifically these are the null space (Theorem NSMS), column space (Theorem CSMS), row space (Theorem RSMS) and left null space (Theorem LNSMS). As subspaces they are vector spaces (Definition S) and it is natural to ask about bases for these vector spaces. Theorem BNS, Theorem BCS, Theorem BRS each have conclusions that provide linearly independent spanning sets for (respectively) the null space, column space, and row space. Notice that each of these theorems contains the word “basis” in its title, even though we did not know the precise meaning of the word at the time. To find a basis for a left null space we can use the definition of this subspace as a null space (Definition LNS) and apply Theorem BNS. Or Theorem FS tells us that the left null space can be expressed as a row space and we can then use Theorem BRS.

Theorem BS is another early result that provides a linearly independent spanning set (i.e. a basis) as its conclusion. If a vector space of column vectors can be expressed as a span of a set of column vectors, then Theorem BS can be employed in a straightforward manner to quickly yield a basis.

SubsectionBSCVBases for Spans of Column Vectors

We have seen several examples of bases in different vector spaces. In this subsection, and the next (Subsection B.BNM), we will consider building bases for \(\complex{m}\) and its subspaces.

Suppose we have a subspace of \(\complex{m}\) that is expressed as the span of a set of vectors, \(S\text{,}\) and \(S\) is not necessarily linearly independent, or perhaps not very attractive. Theorem REMRS says that row-equivalent matrices have identical row spaces, while Theorem BRS says the nonzero rows of a matrix in reduced row-echelon form are a basis for the row space. These theorems together give us a great computational tool for quickly finding a basis for a subspace that is expressed originally as a span.

Example IAS provides another example of this flavor, though now we can notice that \(X\) is a subspace, and that the resulting set of three vectors is a basis. This is such a powerful technique that we should do one more example.

SubsectionBNMBases and Nonsingular Matrices

A quick source of diverse bases for \(\complex{m}\) is the set of columns of a nonsingular matrix.

Proof

Perhaps we should view the fact that the standard unit vectors are a basis (Theorem SUVB) as just a simple corollary of Theorem CNMB? (See Proof Technique LC.)

With a new equivalence for a nonsingular matrix, we can update our list of equivalences.

Proof

SubsectionOBCOrthonormal Bases and Coordinates

We learned about orthogonal sets of vectors in \(\complex{m}\) back in Section O, and we also learned that orthogonal sets are automatically linearly independent (Theorem OSLI). When an orthogonal set also spans a subspace of \(\complex{m}\text{,}\) then the set is a basis. And when the set is orthonormal, then the set is an incredibly nice basis. We will back up this claim with a theorem, but first consider how you might manufacture such a set.

Suppose that \(W\) is a subspace of \(\complex{m}\) with basis \(B\text{.}\) Then \(B\) spans \(W\) and is a linearly independent set of nonzero vectors. We can apply the Gram-Schmidt Procedure (Theorem GSP) and obtain a linearly independent set \(T\) such that \(\spn{T}=\spn{B}=W\) and \(T\) is orthogonal. In other words, \(T\) is a basis for \(W\text{,}\) and is an orthogonal set. By scaling each vector of \(T\) to norm 1, we can convert \(T\) into an orthonormal set, without destroying the properties that make it a basis of \(W\text{.}\) In short, we can convert any basis into an orthonormal basis. Example GSTV, followed by Example ONTV, illustrates this process.

Unitary matrices (Definition UM) are another good source of orthonormal bases (and vice versa). Suppose that \(Q\) is a unitary matrix of size \(n\text{.}\) Then the \(n\) columns of \(Q\) form an orthonormal set (Theorem CUMOS) that is therefore linearly independent (Theorem OSLI). Since \(Q\) is invertible (Theorem UMI), we know \(Q\) is nonsingular (Theorem NI), and then the columns of \(Q\) span \(\complex{n}\) (Theorem CSNM). So the columns of a unitary matrix of size \(n\) are an orthonormal basis for \(\complex{n}\text{.}\)

Why all the fuss about orthonormal bases? Theorem VRRB told us that any vector in a vector space could be written, uniquely, as a linear combination of basis vectors. For an orthonormal basis, finding the scalars for this linear combination is extremely easy, and this is the content of the next theorem. Furthermore, with vectors written this way (as linear combinations of the elements of an orthonormal set) certain computations and analysis become much easier. Here is the promised theorem.

Proof

A slightly less intimidating example follows, in three dimensions and with just real numbers.

Not only do the columns of a unitary matrix form an orthonormal basis, but there is a deeper connection between orthonormal bases and unitary matrices. Informally, the next theorem says that if we transform each vector of an orthonormal basis by multiplying it by a unitary matrix, then the resulting set will be another orthonormal basis. And more remarkably, any matrix with this property must be unitary! As an equivalence (Proof Technique E) we could take this as our defining property of a unitary matrix, though it might not have the same utility as Definition UM.

Proof

SubsectionReading Questions

1

The matrix below is nonsingular. What can you now say about its columns? \begin{equation*} A=\begin{bmatrix} -3 & 0 & 1\\ 1 & 2 & 1\\ 5 & 1 & 6 \end{bmatrix} \end{equation*}

2

Write the vector \(\vect{w}=\colvector{6\\6\\15}\) as a linear combination of the columns of the matrix \(A\) above. How many ways are there to answer this question?

3

Why is an orthonormal basis desirable?

SubsectionExercises

C10

Find a basis for \(\spn{S}\text{,}\) where \begin{align*} S &= \set{ \colvector{1\\3\\2\\1}, \colvector{1\\2\\1\\1}, \colvector{1\\1\\0\\1}, \colvector{1\\2\\2\\1}, \colvector{3\\4\\1\\3} }\text{.} \end{align*}

Solution
C11

Find a basis for the subspace \(W\) of \(\complex{4}\text{,}\) \begin{align*} W &= \setparts{\colvector{a + b - 2c\\a + b - 2c + d\\ -2a + 2b + 4c - d\\ b + d}} {a, b, c, d \in\complexes}\text{.} \end{align*}

Solution
C12

Find a basis for the vector space \(T\) of lower triangular \(3 \times 3\) matrices; that is, matrices of the form \begin{align*} \begin{bmatrix} * & 0 & 0\\ * & * & 0\\ * & * & *\end{bmatrix} \end{align*} where an asterisk represents any complex number.

Solution
C13

Find a basis for the subspace \(Q\) of \(P_2\text{,}\) \(Q = \setparts{p(x) = a + bx + cx^2}{p(0) = 0}\text{.}\)

Solution
C14

Find a basis for the subspace \(R\) of \(P_2\text{,}\) \(R = \setparts{p(x) = a + bx + cx^2}{p'(0) = 0}\text{,}\) where \(p'\) denotes the derivative.

Solution
C40

From Example RSB, form an arbitrary (and nontrivial) linear combination of the four vectors in the original spanning set for \(W\text{.}\) So the result of this computation is of course an element of \(W\text{.}\) As such, this vector should be a linear combination of the basis vectors in \(B\text{.}\) Find the (unique) scalars that provide this linear combination. Repeat with another linear combination of the original four vectors.

Solution
C80

Prove that \(\set{(1,\,2),\,(2,\,3)}\) is a basis for the crazy vector space \(C\) (Example CVS).

M20

In Example BM provide the verifications (linear independence and spanning) to show that \(B\) is a basis of \(M_{mn}\text{.}\)

Solution
T50

Theorem UMCOB says that unitary matrices are characterized as those matrices that “carry” orthonormal bases to orthonormal bases. This problem asks you to prove a similar result: nonsingular matrices are characterized as those matrices that “carry” bases to bases.

More precisely, suppose that \(A\) is a square matrix of size \(n\) and \(B=\set{\vectorlist{x}{n}}\) is a basis of \(\complex{n}\text{.}\) Prove that \(A\) is nonsingular if and only if \(C=\set{A\vect{x}_1,\,A\vect{x}_2,\,A\vect{x}_3,\,\dots,\,A\vect{x}_n}\) is a basis of \(\complex{n}\text{.}\) (See also Exercise PD.T33, Exercise MR.T20.)

Solution
T51

Use the result of Exercise B.T50 to build a very concise proof of Theorem CNMB. (Hint: make a judicious choice for the basis \(B\text{.}\))

Solution