Skip to main content
\(\newcommand{\orderof}[1]{\sim #1} \newcommand{\Z}{\mathbb{Z}} \newcommand{\reals}{\mathbb{R}} \newcommand{\real}[1]{\mathbb{R}^{#1}} \newcommand{\complexes}{\mathbb{C}} \newcommand{\complex}[1]{\mathbb{C}^{#1}} \newcommand{\conjugate}[1]{\overline{#1}} \newcommand{\modulus}[1]{\left\lvert#1\right\rvert} \newcommand{\zerovector}{\vect{0}} \newcommand{\zeromatrix}{\mathcal{O}} \newcommand{\innerproduct}[2]{\left\langle#1,\,#2\right\rangle} \newcommand{\norm}[1]{\left\lVert#1\right\rVert} \newcommand{\dimension}[1]{\dim\left(#1\right)} \newcommand{\nullity}[1]{n\left(#1\right)} \newcommand{\rank}[1]{r\left(#1\right)} \newcommand{\ds}{\oplus} \newcommand{\detname}[1]{\det\left(#1\right)} \newcommand{\detbars}[1]{\left\lvert#1\right\rvert} \newcommand{\trace}[1]{t\left(#1\right)} \newcommand{\sr}[1]{#1^{1/2}} \newcommand{\spn}[1]{\left\langle#1\right\rangle} \newcommand{\nsp}[1]{\mathcal{N}\!\left(#1\right)} \newcommand{\csp}[1]{\mathcal{C}\!\left(#1\right)} \newcommand{\rsp}[1]{\mathcal{R}\!\left(#1\right)} \newcommand{\lns}[1]{\mathcal{L}\!\left(#1\right)} \newcommand{\per}[1]{#1^\perp} \newcommand{\augmented}[2]{\left\lbrack\left.#1\,\right\rvert\,#2\right\rbrack} \newcommand{\linearsystem}[2]{\mathcal{LS}\!\left(#1,\,#2\right)} \newcommand{\homosystem}[1]{\linearsystem{#1}{\zerovector}} \newcommand{\rowopswap}[2]{R_{#1}\leftrightarrow R_{#2}} \newcommand{\rowopmult}[2]{#1R_{#2}} \newcommand{\rowopadd}[3]{#1R_{#2}+R_{#3}} \newcommand{\leading}[1]{\boxed{#1}} \newcommand{\rref}{\xrightarrow{\text{RREF}}} \newcommand{\elemswap}[2]{E_{#1,#2}} \newcommand{\elemmult}[2]{E_{#2}\left(#1\right)} \newcommand{\elemadd}[3]{E_{#2,#3}\left(#1\right)} \newcommand{\scalarlist}[2]{{#1}_{1},\,{#1}_{2},\,{#1}_{3},\,\ldots,\,{#1}_{#2}} \newcommand{\vect}[1]{\mathbf{#1}} \newcommand{\colvector}[1]{\begin{bmatrix}#1\end{bmatrix}} \newcommand{\vectorcomponents}[2]{\colvector{#1_{1}\\#1_{2}\\#1_{3}\\\vdots\\#1_{#2}}} \newcommand{\vectorlist}[2]{\vect{#1}_{1},\,\vect{#1}_{2},\,\vect{#1}_{3},\,\ldots,\,\vect{#1}_{#2}} \newcommand{\vectorentry}[2]{\left\lbrack#1\right\rbrack_{#2}} \newcommand{\matrixentry}[2]{\left\lbrack#1\right\rbrack_{#2}} \newcommand{\lincombo}[3]{#1_{1}\vect{#2}_{1}+#1_{2}\vect{#2}_{2}+#1_{3}\vect{#2}_{3}+\cdots +#1_{#3}\vect{#2}_{#3}} \newcommand{\matrixcolumns}[2]{\left\lbrack\vect{#1}_{1}|\vect{#1}_{2}|\vect{#1}_{3}|\ldots|\vect{#1}_{#2}\right\rbrack} \newcommand{\transpose}[1]{#1^{t}} \newcommand{\inverse}[1]{#1^{-1}} \newcommand{\submatrix}[3]{#1\left(#2|#3\right)} \newcommand{\adj}[1]{\transpose{\left(\conjugate{#1}\right)}} \newcommand{\adjoint}[1]{#1^\ast} \newcommand{\set}[1]{\left\{#1\right\}} \newcommand{\setparts}[2]{\left\lbrace#1\,\middle|\,#2\right\rbrace} \newcommand{\card}[1]{\left\lvert#1\right\rvert} \newcommand{\setcomplement}[1]{\overline{#1}} \newcommand{\charpoly}[2]{p_{#1}\left(#2\right)} \newcommand{\eigenspace}[2]{\mathcal{E}_{#1}\left(#2\right)} \newcommand{\eigensystem}[3]{\lambda&=#2&\eigenspace{#1}{#2}&=\spn{\set{#3}}} \newcommand{\geneigenspace}[2]{\mathcal{G}_{#1}\left(#2\right)} \newcommand{\algmult}[2]{\alpha_{#1}\left(#2\right)} \newcommand{\geomult}[2]{\gamma_{#1}\left(#2\right)} \newcommand{\indx}[2]{\iota_{#1}\left(#2\right)} \newcommand{\ltdefn}[3]{#1\colon #2\rightarrow#3} \newcommand{\lteval}[2]{#1\left(#2\right)} \newcommand{\ltinverse}[1]{#1^{-1}} \newcommand{\restrict}[2]{{#1}|_{#2}} \newcommand{\preimage}[2]{#1^{-1}\left(#2\right)} \newcommand{\rng}[1]{\mathcal{R}\!\left(#1\right)} \newcommand{\krn}[1]{\mathcal{K}\!\left(#1\right)} \newcommand{\compose}[2]{{#1}\circ{#2}} \newcommand{\vslt}[2]{\mathcal{LT}\left(#1,\,#2\right)} \newcommand{\isomorphic}{\cong} \newcommand{\similar}[2]{\inverse{#2}#1#2} \newcommand{\vectrepname}[1]{\rho_{#1}} \newcommand{\vectrep}[2]{\lteval{\vectrepname{#1}}{#2}} \newcommand{\vectrepinvname}[1]{\ltinverse{\vectrepname{#1}}} \newcommand{\vectrepinv}[2]{\lteval{\ltinverse{\vectrepname{#1}}}{#2}} \newcommand{\matrixrep}[3]{M^{#1}_{#2,#3}} \newcommand{\matrixrepcolumns}[4]{\left\lbrack \left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{1}}}\right|\left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{2}}}\right|\left.\vectrep{#2}{\lteval{#1}{\vect{#3}_{3}}}\right|\ldots\left|\vectrep{#2}{\lteval{#1}{\vect{#3}_{#4}}}\right.\right\rbrack} \newcommand{\cbm}[2]{C_{#1,#2}} \newcommand{\jordan}[2]{J_{#1}\left(#2\right)} \newcommand{\hadamard}[2]{#1\circ #2} \newcommand{\hadamardidentity}[1]{J_{#1}} \newcommand{\hadamardinverse}[1]{\widehat{#1}} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \)

SectionMINMMatrix Inverses and Nonsingular Matrices

We saw in Theorem CINM that if a square matrix \(A\) is nonsingular, then there is a matrix \(B\) so that \(AB=I_n\text{.}\) In other words, \(B\) is halfway to being an inverse of \(A\text{.}\) We will see in this section that \(B\) automatically fulfills the second condition (\(BA=I_n\)). Example MWIAA showed us that the coefficient matrix from Archetype A had no inverse. Not coincidentally, this coefficient matrix is singular. We will make all these connections precise now. Not many examples or definitions in this section, just theorems.

SubsectionNMINonsingular Matrices are Invertible

We need a couple of technical results for starters. Some books would call these minor, but essential, results lemmas. We'll just call 'em theorems. See Proof Technique LC for more on the distinction.

The first of these technical results is interesting in that the hypothesis says something about a product of two square matrices and the conclusion then says the same thing about each individual matrix in the product. This result has an analogy in the algebra of complex numbers: suppose \(\alpha,\,\beta\in\complexes\text{,}\) then \(\alpha\beta\neq 0\) if and only if \(\alpha\neq 0\) and \(\beta\neq 0\text{.}\) We can view this result as suggesting that the term “nonsingular” for matrices is like the term “nonzero” for scalars. Consider too that we know singular matrices, as coefficient matrices for systems of equations, will sometimes lead to systems with no solutions, or systems with infinitely many solutions (Theorem NMUS). What do linear equations with zero look like? Consider \(0x=5\text{,}\) which has no solution, and \(0x=0\text{,}\) which has infinitely many solutions. In the algebra of scalars, zero is exceptional (meaning different, not better), and in the algebra of matrices, singular matrices are also the exception. While there is only one zero scalar, and there are infinitely many singular matrices, we will see that singular matrices are a distinct minority.

Proof

This is a powerful result in the “forward” direction, because it allows us to begin with a hypothesis that something complicated (the matrix product \(AB\)) has the property of being nonsingular, and we can then conclude that the simpler constituents (\(A\) and \(B\) individually) then also have the property of being nonsingular. If we had thought that the matrix product was an artificial construction, results like this would make us begin to think twice.

The contrapositive of this entire result is equally interesting. It says that \(A\) or \(B\) (or both) is a singular matrix if and only if the product \(AB\) is singular. (See Proof Technique CP.)

Proof

So Theorem OSIS tells us that if \(A\) is nonsingular, then the matrix \(B\) guaranteed by Theorem CINM will be both a “right-inverse” and a “left-inverse” for \(A\text{,}\) so \(A\) is invertible and \(\inverse{A}=B\text{.}\)

So if you have a nonsingular matrix, \(A\text{,}\) you can use the procedure described in Theorem CINM to find an inverse for \(A\text{.}\) If \(A\) is singular, then the procedure in Theorem CINM will fail as the first \(n\) columns of \(M\) will not row-reduce to the identity matrix. However, we can say a bit more. When \(A\) is singular, then \(A\) does not have an inverse (which is very different from saying that the procedure in Theorem CINM fails to find an inverse). This may feel like we are splitting hairs, but it is important that we do not make unfounded assumptions. These observations motivate the next theorem.

Proof

So for a square matrix, the properties of having an inverse and of having a trivial null space are one and the same. Cannot have one without the other.

Proof

In the case that \(A\) is a nonsingular coefficient matrix of a system of equations, the inverse allows us to very quickly compute the unique solution, for any vector of constants.

Proof

SubsectionUMUnitary Matrices

Recall that the adjoint of a matrix is \(\adjoint{A}=\transpose{\left(\conjugate{A}\right)}\) (Definition A).

DefinitionUMUnitary Matrices

Suppose that \(U\) is a square matrix of size \(n\) such that \(\adjoint{U}U=I_n\text{.}\) Then we say \(U\) is unitary.

This condition may seem rather far-fetched at first glance. Would there be any matrix that behaved this way? Well, yes, here is one.

Unitary matrices do not have to look quite so gruesome. Here is a larger one that is a bit more pleasing.

If a matrix \(A\) has only real number entries (we say it is a real matrix) then the defining property of being unitary simplifies to \(\transpose{A}A=I_n\text{.}\) In this case we, and everybody else, call the matrix orthogonal, so you may often encounter this term in your other reading when the complex numbers are not under consideration.

Unitary matrices have easily computed inverses. They also have columns that form orthonormal sets. Here are the theorems that show us that unitary matrices are not as strange as they might initially appear.

Proof
Proof

When using vectors and matrices that only have real number entries, orthogonal matrices are those matrices with inverses that equal their transpose. Similarly, the inner product is the familiar dot product. Keep this special case in mind as you read the next theorem.

Proof

Aside from the inherent interest in this theorem, it makes a bigger statement about unitary matrices. When we view vectors geometrically as directions or forces, then the norm equates to a notion of length. If we transform a vector by multiplication with a unitary matrix, then the length (norm) of that vector stays the same. If we consider column vectors with two or three slots containing only real numbers, then the inner product of two such vectors is just the dot product, and this quantity can be used to compute the angle between two vectors. When two vectors are multiplied (transformed) by the same unitary matrix, their dot product is unchanged and their individual lengths are unchanged. This results in the angle between the two vectors remaining unchanged.

A unitary transformation (matrix-vector products with unitary matrices) thus preserve geometrical relationships among vectors representing directions, forces, or other physical quantities. In the case of a two-slot vector with real entries, this is simply a rotation. These sorts of computations are exceedingly important in computer graphics such as games and real-time simulations, especially when increased realism is achieved by performing many such computations quickly. We will see unitary matrices again in subsequent sections (especially Theorem OD) and in each instance, consider the interpretation of the unitary matrix as a sort of geometry-preserving transformation. Some authors use the term isometry to highlight this behavior. We will speak loosely of a unitary matrix as being a sort of generalized rotation.

A final reminder: the terms “dot product,” “symmetric matrix” and “orthogonal matrix” used in reference to vectors or matrices with real number entries are special cases of the terms “inner product,” “Hermitian matrix” and “unitary matrix” that we use for vectors or matrices with complex number entries, so keep that in mind as you read elsewhere.

SubsectionReading Questions

1

Compute the inverse of the coefficient matrix of the system of equations below and use the inverse to solve the system. \begin{align*} 4x_1 + 10x_2 &= 12\\ 2x_1 + 6x_2 &= 4 \end{align*}

2

In the reading questions for Section MISLE you were asked to find the inverse of the \(3\times 3\) matrix below. \begin{equation*} \begin{bmatrix} 2 & 3 & 1\\ 1 & -2 & -3\\ -2 & 4 & 6 \end{bmatrix} \end{equation*} Because the matrix was not nonsingular, you had no theorems at that point that would allow you to compute the inverse. Explain why you now know that the inverse does not exist (which is different than not being able to compute it) by quoting the relevant theorem's acronym.

3

Is the matrix \(A\) unitary? Why? \begin{equation*} A=\begin{bmatrix} \frac{1}{\sqrt{22}}\left(4+2i\right) & \frac{1}{\sqrt{374}}\left(5+3i\right) \\ \frac{1}{\sqrt{22}}\left(-1-i\right) & \frac{1}{\sqrt{374}}\left(12+14i\right) \\ \end{bmatrix} \end{equation*}

SubsectionExercises

C20

Verify that \(AB\) is nonsingular. \begin{align*} A&= \begin{bmatrix} 1 & 2 & 1 \\ 0 & 1 & 1\\ 1 & 0 & 2 \end{bmatrix} & B &= \begin{bmatrix} -1 & 1 & 0\\ 1 & 2 & 1\\ 0 & 1 & 1 \end{bmatrix} \end{align*}

C40

Solve the system of equations below using the inverse of a matrix. \begin{align*} x_1+x_2+3x_3+x_4&=5\\ -2x_1-x_2-4x_3-x_4&=-7\\ x_1+4x_2+10x_3+2x_4&=9\\ -2x_1-4x_3+5x_4&=9 \end{align*}

Solution
M10

Find values of \(x\text{,}\) \(y\text{,}\) \(z\) so that matrix \(A\) is invertible. \begin{equation*} A = \begin{bmatrix} 1 & 2 & x\\ 3 & 0 & y \\ 1 & 1 & z \end{bmatrix} \end{equation*}

Solution
M11

Find values of \(x\text{,}\) \(y\) \(z\) so that matrix \(A\) is singular. \begin{equation*} A = \begin{bmatrix} 1 & x & 1\\ 1 & y & 4 \\ 0 & z & 5 \end{bmatrix} \end{equation*}

Solution
M15

If \(A\) and \(B\) are \(n \times n\) matrices, \(A\) is nonsingular, and \(B\) is singular, show directly that \(AB\) is singular, without using Theorem NPNT.

Solution
M20

Construct an example of a \(4\times 4\) unitary matrix.

Solution
M80

Matrix multiplication interacts nicely with many operations. But not always with transforming a matrix to reduced row-echelon form. Suppose that \(A\) is an \(m\times n\) matrix and \(B\) is an \(n\times p\) matrix. Let \(P\) be a matrix that is row-equivalent to \(A\) and in reduced row-echelon form, \(Q\) be a matrix that is row-equivalent to \(B\) and in reduced row-echelon form, and let \(R\) be a matrix that is row-equivalent to \(AB\) and in reduced row-echelon form. Is \(PQ=R\text{?}\) (In other words, with nonstandard notation, is \(\text{rref}(A)\text{rref}(B)=\text{rref}(AB)\text{?}\))

Construct a counterexample to show that, in general, this statement is false. Then find a large class of matrices where if \(A\) and \(B\) are in the class, then the statement is true.

Solution
T10

Suppose that \(Q\) and \(P\) are unitary matrices of size \(n\text{.}\) Prove that \(QP\) is a unitary matrix.

T11

Prove that Hermitian matrices (Definition HM) have real entries on the diagonal. More precisely, suppose that \(A\) is a Hermitian matrix of size \(n\text{.}\) Then \(\matrixentry{A}{ii}\in\reals\text{,}\) \(1\leq i\leq n\text{.}\)

T12

Suppose that we are checking if a square matrix of size \(n\) is unitary. Show that a straightforward application of Theorem CUMOS requires the computation of \(n^2\) inner products when the matrix is unitary, and fewer when the matrix is not orthogonal. Then show that this maximum number of inner products can be reduced to \(\frac{1}{2}n(n+1)\) in light of Theorem IPAC.

T25

The notation \(A^k\) means a repeated matrix product between \(k\) copies of the square matrix \(A\text{.}\)

  1. Assume \(A\) is an \(n\times n\) matrix where \(A^2=\zeromatrix\) (which does not imply that \(A=\zeromatrix\text{.}\)) Prove that \(I_n-A\) is invertible by showing that \(I_n+A\) is an inverse of \(I_n-A\text{.}\)
  2. Assume that \(A\) is an \(n\times n\) matrix where \(A^3=\zeromatrix\text{.}\) Prove that \(I_n-A\) is invertible.
  3. Form a general theorem based on your observations from parts (1) and (2) and provide a proof.