Section B  Bases

From A First Course in Linear Algebra
Version 1.04
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/

A basis of a vector space is one of the most useful concepts in linear algebra. It often provides a finite description of an infinite vector space.

Subsection B: Bases

We now have all the tools in place to define a basis of a vector space.

Definition B
Basis
Suppose $V$ is a vector space. Then a subset $S\subseteq V$ is a basis of $V$ if it is linearly independent and spans $V$. $△$

So, a basis is a linearly independent spanning set for a vector space. The requirement that the set spans $V$ insures that $S$ has enough raw material to build $V$, while the linear independence requirement insures that we do not have any more raw material than we need. As we shall see soon in Section D, a basis is a minimal spanning set.

You may have noticed that we used the term basis for some of the titles of previous theorems (e.g. Theorem BNS, Theorem BCS, Theorem BRS) and if you review each of these theorems you will see that their conclusions provide linearly independent spanning sets for sets that we now recognize as subspaces of ${ℂ}^m$. Examples associated with these theorems include Example NSLIL, Example CSOCD and Example IAS. As we will see, these three theorems will continue to be powerful tools, even in the setting of more general vector spaces.

Furthermore, the archetypes contain an abundance of bases. For each coefficient matrix of a system of equations, and for each archetype defined simply as a matrix, there is a basis for the null space, three bases for the column space, and a basis for the row space. For this reason, our subsequent examples will concentrate on bases for vector spaces other than ${ℂ}^m$. Notice that Definition B does not preclude a vector space from having many bases, and this is the case, as hinted above by the statement that the archetypes contain three bases for the column space of a matrix. More generally, we can grab any basis for a vector space, multiply any one basis vector by a non-zero scalar and create a slightly different set that is still a basis. For “important” vector spaces, it will be convenient to have a collection of “nice” bases. When a vector space has a single particularly nice basis, it is sometimes called the standard basis though there is nothing precise enough about this term to allow us to define it formally — it is a question of style. Here are some nice bases for important vector spaces.

Theorem SUVB
Standard Unit Vectors are a Basis
The set of standard unit vectors for ${ℂ}^m$ (Definition SUV), $B=\left\{e_1,e_2,e_3,\dots ,e_m\right\}=\left\{\left.e_i\right|1\le i\le m\right\}$ is a basis for the vector space ${ℂ}^m$. $\square$

Proof   We must show that the set $B$ is both linearly independent and a spanning set for ${ℂ}^m$. First, the vectors in $B$ are, by Definition SUV, the columns of the identity matrix, which we know is nonsingular (since it row-reduces to the identity matrix, Theorem NMRRI). And the columns of a nonsingular matrix are linearly independent by Theorem NMLIC.

Suppose we grab an arbitrary vector from ${ℂ}^m$, say

Can we write $v$ as a linear combination of the vectors in $B$? Yes, and quite simply.

this shows that ${ℂ}^m\subseteq 〈B〉$, which is sufficient to show that $B$ is a spanning set for ${ℂ}^m$. $■$

Example BP
Bases for $P_n$
The vector space of polynomials with degree at most $n$, $P_n$, has the basis

Another nice basis for $P_n$ is

Checking that each of $B$ and $C$ is a linearly independent spanning set are good exercises. $⊠$

Example BM
A basis for the vector space of matrices
In the vector space $M_{mn}$ of matrices (Example VSM) define the matrices $B_{k\ell }$, $1\le k\le m$, $1\le \ell \le n$ by

So these matrices have entries that are all zeros, with the exception of a lone entry that is one. The set of all $mn$ of them,

forms a basis for $M_{mn}$. $⊠$

The bases described above will often be convenient ones to work with. However a basis doesn’t have to obviously look like a basis.

Example BSP4
A basis for a subspace of $P_4$
In Example SSP4 we showed that

is a spanning set for $W=\left\{\left.p\left(x\right)\right|p\in P_4,p\left(2\right)=0\right\}$. We will now show that $S$ is also linearly independent in $W$. Begin with a relation of linear dependence,

Equating coefficients (vector equality in $P_4$) gives the homogeneous system of five equations in four variables,

We form the coefficient matrix, and row-reduce to obtain a matrix in reduced row-echelon form

With only the trivial solution to this homogeneous system, we conclude that only scalars that will form a relation of linear dependence are the trivial ones, and therefore the set $S$ is linearly independent (Definition LI). Finally, $S$ has earned the right to be called a basis for $W$ (Definition B). $⊠$

Example BSM22
A basis for a subspace of $M_{22}$
In Example SSM22 we discovered that

is a spanning set for the subspace

of the vector space of all $2\times 2$ matrices, $M_{22}$. If we can also determine that $Q$ is linearly independent in $Z$ (or in $M_{22}$), then it will qualify as a basis for $Z$. Let’s begin with a relation of linear dependence.

Using our definition of matrix equality (Definition ME) we equate corresponding entries and get a homogeneous system of four equations in two variables,

We could row-reduce the coefficient matrix of this homogeneous system, but it is not necessary. The second and fourth equations tell us that ${\alpha }_1=0$, ${\alpha }_2=0$ is the only solution to this homogeneous system. This qualifies the set $Q$ as being linearly independent, since the only relation of linear dependence is trivial (Definition LI). Therefore $Q$ is a basis for $Z$ (Definition B). $⊠$

Example BC
Basis for the crazy vector space
In Example LIC and Example SSC we determined that the set $R=\left\{\left(1,0\right),\left(6,3\right)\right\}$ from the crazy vector space, $C$ (Example CVS), is linearly independent and is a spanning set for $C$. By Definition B we see that $R$ is a basis for $C$. $⊠$

We have seen that several of the sets associated with a matrix are subspaces of vector spaces of column vectors. Specifically these are the null space (Theorem NSMS), column space (Theorem CSMS), row space (Theorem RSMS) and left null space (Theorem LNSMS). As subspaces they are vector spaces (Definition S) and it is natural to ask about bases for these vector spaces. Theorem BNS, Theorem BCS, Theorem BRS each have conclusions that provide linearly independent spanning sets for (respectively) the null space, column space, and row space. Notice that each of these theorems contains the word “basis” in its title, even though we did not know the precise meaning of the word at the time. To find a basis for a left null space we can use the definition of this subspace as a null space (Definition LNS) and apply Theorem BNS. Or Theorem FS tells us that the left null space can be expressed as a row space and we can then use Theorem BRS.

Theorem BS is another early result that provides a linearly independent spanning set (i.e. a basis) as its conclusion. If a vector space of column vectors can be expressed as a span of a set of column vectors, then Theorem BS can be employed in a straightforward manner to quickly yield a basis.

Subsection BSCV: Bases for Spans of Column Vectors

We have seen several examples of bases in different vector spaces. In this subsection, and the next (Subsection B.BNM), we will consider building bases for ${ℂ}^m$ and its subspaces.

Suppose we have a subspace of ${ℂ}^m$ that is expressed as the span of a set of vectors, $S$, and $S$ is not necessarily linearly independent, or perhaps not very attractive. Theorem REMRS says that row-equivalent matrices have identical row spaces, while Theorem BRS says the nonzero rows of a matrix in reduced row-echelon form are a basis for the row space. These theorems together give us a great computational tool for quickly finding a basis for a subspace that is expressed originally as a span.

Example RSB
Row space basis
When we first defined the span of a set of column vectors, in Example SCAD we looked at the set

with an eye towards realizing $W$ as the span of a smaller set. By building relations of linear dependence (though we did not know them by that name then) we were able to remove two vectors and write $W$ as the span of the other two vectors. These two remaining vectors formed a linearly independent set, even though we did not know that at the time.

Now we know that $W$ is a subspace and must have a basis. Consider the matrix, $C$, whose rows are the vectors in the spanning set for $W$,

Then, by Definition RSM, the row space of $C$ will be $W$, $\mathcal{ℛ}\left(C\right)=W$. Theorem BRS tells us that if we row-reduce $C$, the nonzero rows of the row-equivalent matrix in reduced row-echelon form will be a basis for $\mathcal{ℛ}\left(C\right)$, and hence a basis for $W$. Let’s do it — $C$ row-reduces to

If we convert the two nonzero rows to column vectors then we have a basis,

and

For aesthetic reasons, we might wish to multiply each vector in $B$ by $11$, which will not change the spanning or linear independence properties of $B$ as a basis. Then we can also write

Example IAS provides another example of this flavor, though now we can notice that $X$ is a subspace, and that the resulting set of three vectors is a basis. This is such a powerful technique that we should do one more example.

Example RS
Reducing a span
In Example RSC5 we began with a set of $n=4$ vectors from ${ℂ}^5$,

and defined $V=〈R〉$. Our goal in that problem was to find a relation of linear dependence on the vectors in $R$, solve the resulting equation for one of the vectors, and re-express $V$ as the span of a set of three vectors.

Here is another way to accomplish something similar. The row space of the matrix

is equal to $〈R〉$. By Theorem BRS we can row-reduce this matrix, ignore any zero rows, and use the non-zero rows as column vectors that are a basis for the row space of $A$. Row-reducing $A$ creates the matrix

So

is a basis for $V$. Our theorem tells us this is a basis, there is no need to verify that the subspace spanned by three vectors (rather than four) is the identical subspace, and there is no need to verify that we have reached the limit in reducing the set, since the set of three vectors is guaranteed to be linearly independent. $⊠$

Subsection BNM: Bases and Nonsingular Matrices

A quick source of diverse bases for ${ℂ}^m$ is the set of columns of a nonsingular matrix.

Theorem CNMB
Columns of Nonsingular Matrix are a Basis
Suppose that $A$ is a square matrix of size $m$. Then the columns of $A$ are a basis of ${ℂ}^m$ if and only if $A$ is nonsingular. $\square$

Proof   ($\Rightarrow$) Suppose that the columns of $A$ are a basis for ${ℂ}^m$. Then Definition B says the set of columns is linearly independent. Theorem NMLIC then says that $A$ is nonsingular.

($\Leftarrow$) Suppose that $A$ is nonsingular. Then by Theorem NMLIC this set of columns is linearly independent. Theorem CSNM says that for a nonsingular matrix, $\mathcal{C}\left(A\right)={ℂ}^m$. This is equivalent to saying that the columns of $A$ are a spanning set for the vector space ${ℂ}^m$. As a linearly independent spanning set, the columns of $A$ qualify as a basis for ${ℂ}^m$ (Definition B). $■$

Example CABAK
Columns as Basis, Archetype K
Archetype K is the $5\times 5$ matrix

which is row-equivalent to the $5\times 5$ identity matrix $I_5$. So by Theorem NMRRI, $K$ is nonsingular. Then Theorem CNMB says the set

is a (novel) basis of ${ℂ}^5$. $⊠$

Perhaps we should view the fact that the standard unit vectors are a basis (Theorem SUVB) as just a simple corollary of Theorem CNMB? (See Technique LC.)

With a new equivalence for a nonsingular matrix, we can update our list of equivalences.

Theorem NME5
Nonsingular Matrix Equivalences, Round 5
Suppose that $A$ is a square matrix of size $n$. The following are equivalent.

1. $A$ is nonsingular.
2. $A$ row-reduces to the identity matrix.
3. The null space of $A$ contains only the zero vector, $\mathcal{N}\left(A\right)=\left\{0\right\}$.
4. The linear system $\mathcal{ℒ}\mathcal{S}\left(A,b\right)$ has a unique solution for every possible choice of $b$.
5. The columns of $A$ are a linearly independent set.
6. $A$ is invertible.
7. The column space of $A$ is ${ℂ}^n$, $\mathcal{C}\left(A\right)={ℂ}^n$.
8. The columns of $A$ are a basis for ${ℂ}^n$.

Proof   With a new equivalence for a nonsingular matrix in Theorem CNMB we can expand Theorem NME4. $■$

Subsection OBC: Orthonormal Bases and Coordinates

We learned about orthogonal sets of vectors in ${ℂ}^m$ back in Section O, and we also learned that orthogonal sets are automatically linearly independent (Theorem OSLI). When an orthogonal set also spans a subspace of ${ℂ}^m$, then the set is a basis. And when the set is orthonormal, then the set is an incredibly nice basis. We will back up this claim with a theorem, but first consider how you might manufacture such a set.

Suppose that $W$ is a subspace of ${ℂ}^m$ with basis $B$. Then $B$ spans $W$ and is a linearly independent set of nonzero vectors. We can apply the Gram-Schmidt Procedure (Theorem GSPCV) and obtain a linearly independent set $T$ such that $〈T〉=〈B〉=W$ and $T$ is orthogonal. In other words, $T$ is a basis for $W$, and is an orthogonal set. By scaling each vector of $T$ to norm 1, we can convert $T$ into an orthonormal set, without destroying the properties that make it a basis of $W$. In short, we can convert any basis into an orthonormal basis. Example GSTV, followed by Example ONTV, illustrates this process.

Unitary matrices (Definition UM) are another good source of orthonormal bases (and vice versa). Suppose that $Q$ is a unitary matrix of size $n$. Then the $n$ columns of $Q$ form an orthonormal set (Theorem CUMOS) that is therefore linearly independent (Theorem OSLI). Since $Q$ is invertible (Theorem UMI), we know $Q$ is nonsingular (Theorem NI), and then the columns of $Q$ span ${ℂ}^n$ (Theorem CSNM). So the columns of a unitary matrix of size $n$ are an orthonormal basis for ${ℂ}^n$.

Why all the fuss about orthonormal bases? Theorem VRRB told us that any vector in a vector space could be written, uniquely, as a linear combination of basis vectors. For an orthonormal basis, finding the scalars for this linear combination is extremely easy, and this is the content of the next theorem. Furthermore, with vectors written this way (as linear combinations of the elements of an orthonormal set) certain computations and analysis become much easier. Here’s the promised theorem.

Theorem COB
Coordinates and Orthonormal Bases
Suppose that $B=\left\{v_1,v_2,v_3,\dots ,v_p\right\}$ is an orthonormal basis of the subspace $W$ of ${ℂ}^m$. For any $w\in W$,

Proof   Because $B$ is a basis of $W$, Theorem VRRB tells us that we can write $w$ uniquely as a linear combination of the vectors in $B$. So it is not this aspect of the conclusion that makes this theorem interesting. What is interesting is that the particular scalars are so easy to compute. No need to solve big systems of equations — just do an inner product of $w$ with $v_i$ to arrive at the coefficient of $v_i$ in the linear combination.

So begin the proof by writing $w$ as a linear combination of the vectors in $B$, using unknown scalars,

and compute,

So the (unique) scalars for the linear combination are indeed the inner products advertised in the conclusion of the theorem’s statement. $■$

Example CROB4
Coordinatization relative to an orthonormal basis, ${ℂ}^4$
The set

was proposed, and partially verified, as an orthogonal set in Example AOS. Let’s scale each vector to norm 1, so as to form an orthonormal basis of ${ℂ}^4$. (Notice that by Theorem OSLI the set is linearly independent. Since we know the dimension of ${ℂ}^4$ is 4, Theorem G tells us the set is just the right size to be a basis of ${ℂ}^4$.) The norms of these vectors are,

So an orthonormal basis is

Now, choose any vector from ${ℂ}^4$, say $w=\left[\begin{array}{c}\hfill 2\hfill \\ \hfill -3\hfill \\ \hfill 1\hfill \\ \hfill 4\hfill \end{array}\right]$, and compute

then Theorem COB guarantees that

as you might want to check (if you have unlimited patience). $⊠$

A slightly less intimidating example follows, in three dimensions and with just real numbers.

Example CROB3
Coordinatization relative to an orthonormal basis, ${ℂ}^3$
The set

is a linearly independent set, which the Gram-Schmidt Process (Theorem GSPCV) converts to an orthogonal set, and which can then be converted to the orthonormal set,

which is therefore an orthonormal basis of ${ℂ}^3$. With three vectors in ${ℂ}^3$, all with real number entries, the inner product (Definition IP) reduces to the usual “dot product” (or scalar product) and the orthogonal pairs of vectors can be interpreted as perpendicular pairs of directions. So the vectors in $B$ serve as replacements for our usual 3-D axes, or the usual 3-D unit vectors $\overrightarrow{i},\overrightarrow{j}$ and $\overrightarrow{k}$. We would like to decompose arbitrary vectors into “components” in the directions of each of these basis vectors. It is Theorem COB that tells us how to do this.

Suppose that we choose $w=\left[\begin{array}{c}\hfill 2\hfill \\ \hfill -1\hfill \\ \hfill 5\hfill \end{array}\right]$. Compute

then Theorem COB guarantees that

which you should be able to check easily, even if you do not have much patience. $⊠$

Subsection READ: Reading Questions

1. The matrix below is nonsingular. What can you now say about its columns?

   $$A=\left[\begin{array}{ccc}\hfill -3\hfill & \hfill 0\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 2\hfill & \hfill 1\hfill \\ \hfill 5\hfill & \hfill 1\hfill & \hfill 6\hfill \end{array}\right]$$

2. Write the vector $w=\left[\begin{array}{c}\hfill 6\hfill \\ \hfill 6\hfill \\ \hfill 15\hfill \end{array}\right]$ as a linear combination of the columns of the matrix $A$ above. How many ways are there to answer this question?
3. Why is an orthonormal basis desirable?

Subsection EXC: Exercises

C40 From Example RSB, form an arbitrary (and nontrivial) linear combination of the four vectors in the original spanning set for $W$. So the result of this computation is of course an element of $W$. As such, this vector should be a linear combination of the basis vectors in $B$. Find the (unique) scalars that provide this linear combination. Repeat with another linear combination of the original four vectors.  
Contributed by Robert Beezer Solution [939]

C80 Prove that $\left\{\left(1,2\right),\left(2,3\right)\right\}$ is a basis for the crazy vector space $C$ (Example CVS).  
Contributed by Robert Beezer

M20 In Example BM provide the verifications (linear independence and spanning) to show that $B$ is a basis of $M_{mn}$.  
Contributed by Robert Beezer Solution [936]

Subsection SOL: Solutions

M20 Contributed by Robert Beezer Statement [935]
We need to establish the linear independence and spanning properties of the set

relative to the vector space $M_{mn}$.

This proof is more transparent if you write out individual matrices in the basis with lots of zeros and dots and a lone one. But we don’t have room for that here, so we will use summation notation. Think carefully about each step, especially when the double summations seem to “disappear.” Begin with a relation of linear dependence, using double subscripts on the scalars to align with the basis elements.

Now consider the entry in row $i$ and column $j$ for these equal matrices,

Since $i$ and $j$ were arbitrary, we find that each scalar is zero and so $B$ is linearly independent (Definition LI).

To establish the spanning property of $B$ we need only show that an arbitrary matrix $A$ can be written as a linear combination of the elements of $B$. So suppose that $A$ is an arbitrary $m\times n$ matrix and consider the matrix $C$ defined as a linear combination of the elements of $B$ by

Then,

So by Definition ME, $A=C$, and therefore $A\in 〈B〉$. By Definition B, the set $B$ is a basis of the vector space $M_{mn}$.

C40 Contributed by Robert Beezer Statement [935]
An arbitrary linear combination is

(You probably used a different collection of scalars.) We want to write $y$ as a linear combination of

We could set this up as vector equation with variables as scalars in a linear combination of the vectors in $B$, but since the first two slots of $B$ have such a nice pattern of zeros and ones, we can determine the necessary scalars easily and then double-check our answer with a computation in the third slot,

Notice how the uniqueness of these scalars arises. They are forced to be $25$ and $-10$.