SCLA Orthogonal Complements

Section 1.3 Orthogonal Complements

Theorem (((above on repeated sums))) mentions repeated sums, which are of interest. However, when we begin with a vector space

$V$ and a single subspace

$W\text{,}$ we can ask about the existence of another subspace,

$W\text{,}$ such that

$V=U\ds W\text{.}$ The answer is that such a

$W$ always exists, and we then refer to it as a complement of

$U\text{.}$

🔗

Definition 1.3.1. Subspace Complement.

Suppose that $V$ is a vector space with a subspace $U\text{.}$ If $W$ is a subspace such that $V=U\ds W\text{,}$ then $W$ is the complement of $V\text{.}$

🔗

Every subspace has a complement, and generally it is not unique.

🔗

Lemma 1.3.2. Every Subspace has a Complement.

Suppose that $V$ is a vector space with a subspace $U\text{.}$ Then there exists a subspace $W$ such that $V=U\ds W\text{,}$ so $W$ is the complement of $V\text{.}$

🔗

Proof.

Suppose that $\dimension{V}=n$ and $\dimension{U}=k\text{,}$ and let $B=\set{\vectorlist{u}{k}}$ be a basis of $U\text{.}$ With $n-k$ applications of Theorem ELIS we obtain vectors $\vectorlist{v}{n-k}$ that succesively create bases $B_i=\set{\vectorlist{u}{k},\,\vectorlist{v}{i}}\text{,}$ $0\leq i\leq n-k$ for subspaces $U=U_0, U_1,\dots,U_{n-k}=V\text{,}$ where $\dimension{U_i}=k+i\text{.}$

Define $W=\spn{\set{\vectorlist{v}{n-k}}}\text{.}$ Since $\set{\vectorlist{u}{k},\,\vectorlist{v}{i}}$ is a basis for $V$ and $\set{\vectorlist{u}{k}}$ is a basis for $U\text{,}$ we can apply Theorem (((Direct Sum From a Basis (above)))) to see that $V=U\ds W\text{,}$ so $W$ is the complement of $V\text{.}$ (Compare with (((Direct Sum From One Subspace (above)))), which has identical content, but a different write-up.)

🔗

The freedom given when we “extend” a linearly independent set (or basis) to create a basis for the complement means that we can create a complement in many ways, so it is not unique.

🔗

Checkpoint 1.3.3.

Consider the subspace $U$ of $V=\complex{3}\text{,}$

\begin{equation*} U=\spn{\set{\colvector{1\\-6\\-8},\colvector{1\\-5\\-7}}}. \end{equation*}

Create two different complements of $U\text{,}$ being sure to prove that your complements are unequal (and not simply have unequal bases). Before reading ahead, can you think of an ideal (or “canonical”) choice for the complement?

🔗

Checkpoint 1.3.4.

Consider the subspace $U$ of $V=\complex{5}\text{,}$

\begin{equation*} U=\spn{\set{\colvector{1\\-4\\-2\\6\\-5},\colvector{1\\-4\\-1\\4\\-3}}}. \end{equation*}

Create a complement of $U\text{.}$ (If you have read ahead, do not create an orthogonal complement for this exercise.)

🔗

With an inner product, and a notion of orthogonality, we can define a canonical, and useful, complement for every subspace.

🔗

Definition 1.3.5. Orthogonal Complement.

Suppose that $V$ is a vector space with a subspace $U\text{.}$ Then the orthogonal complement of $U$ (relative to $V$ ) is

$\begin{equation*} \per{U} = \setparts{\vect{v}\in V}{\innerproduct{\vect{v}}{\vect{u}}=0\text{ for every }\vect{u}\in U}. \end{equation*}$

🔗

A matrix formulation of the orthogonal complement will help us establish that the moniker “complement” is deserved.

🔗

Theorem 1.3.6. Orthogonal Complement as a Null Space.

Suppose that $V$ is a vector space with a subspace $U\text{.}$ Let $A$ be a matrix whose columns are a spanning set for $U\text{.}$ Then $\per{U}=\nsp{\adjoint{A}}\text{.}$

🔗

Proof.

Membership in the orthogonal complement requires a vector to be orthogonal to every vector of $U\text{.}$ However, because of the linearity of the inner product (Theorem IPVA, Theorem IPSM), it is equivalent to require that a vector be orthogonal to each member of a spanning set for $U\text{.}$ So membership in the orthogonal complement is equivalent to being orthogonal to each column of $A\text{.}$ We obtain the desired set equality from the equivalences,

\begin{equation*} \vect{v}\in\per{U} \iff \adjoint{\vect{v}}A=\adjoint{\zerovector} \iff \adjoint{A}\vect{v}=\zerovector \iff \vect{v}\in\nsp{\adjoint{A}}. \end{equation*}

🔗

Theorem 1.3.7. Orthogonal Complement Decomposition.

Suppose that $V$ is a vector space with a subspace $U\text{.}$ Then $V=U\ds\per{U}\text{.}$

🔗

Proof.

We first establish that $U\cap\per{U}=\set{\zerovector}\text{.}$ Suppose $\vect{u}\in U$ and $\vect{u}\in\per{U}\text{.}$ Then $\innerproduct{\vect{u}}{\vect{u}}=0$ and by Theorem PIP we conclude that $\vect{u}=\zerovector\text{.}$

We now show that an arbitrary vector $\vect{v}$ can be written as a sum of vectors from $U$ and $\per{U}\text{.}$ Without loss of generality, we can assume we have an orthonormal basis for $U\text{,}$ for if not, we can apply the Gram-Schmidt process to any basis of $U$ to create an orthogonal spanning set, whose individual vectors can be scaled to have norm one (Theorem GSP). Denote this basis as $B=\set{\vectorlist{u}{k}}\text{.}$

Define the vector $\vect{v}_1$ as a linear combination of the vectors of $B\text{,}$ so $\vect{v}_1\in U\text{.}$

\begin{equation*} \vect{v}_1 = \sum_{i=1}^k\innerproduct{\vect{u}_i}{\vect{v}}\,\vect{u}_i\text{.} \end{equation*}

Define $\vect{v}_2 = \vect{v}-\vect{v}_1\text{,}$ so trivially by construction, $\vect{v}=\vect{v}_1+\vect{v}_2\text{.}$ It remains to show that $\vect{v}_2\in\per{U}\text{.}$ We repeatedly use properties of the inner product. This construction and proof may remind you of the Gram-Schmidt process. For $1\leq j\leq k\text{,}$

\begin{align*} \innerproduct{\vect{v}_2}{\vect{u}_j} &=\innerproduct{\vect{v}}{\vect{u}_j}-\innerproduct{\vect{v}_1}{\vect{u}_j}\\ &=\innerproduct{\vect{v}}{\vect{u}_j}-\sum_{i=1}^k\innerproduct{\innerproduct{\vect{u}_i}{\vect{v}}\,\vect{u}_i}{\vect{u}_j}\\ &=\innerproduct{\vect{v}}{\vect{u}_j}-\sum_{i=1}^k\conjugate{\innerproduct{\vect{u}_i}{\vect{v}}}\innerproduct{\vect{u}_i}{\vect{u}_j}\\ &=\innerproduct{\vect{v}}{\vect{u}_j}-\conjugate{\innerproduct{\vect{u}_j}{\vect{v}}}\innerproduct{\vect{u}_j}{\vect{u}_j}\\ &=\innerproduct{\vect{v}}{\vect{u}_j}-\innerproduct{\vect{v}}{\vect{u}_j}\\ &=0 \end{align*}

We have fulfilled the hypotheses of Theorem Theorem 1.2.5 and so can say $V=U\ds\per{U}\text{.}$

🔗

Theorem Theorem 1.3.7 gives us a canonical choice of a complementary subspace, which has useful orthogonality properties. It also allows us to decompose any vector (uniquely) into an element of a subspace, plus an orthogonal vector. This might remind you in some ways of “resolving a vector into compoments” if you have studied physics some.

🔗

Given a matrix, we get a natural vector space decomposition.

🔗

Corollary 1.3.8. Matrix Subspace Decomposition.

Suppose that $A$ is an $m\times n$ matrix. Then

$\begin{equation*} \complex{m}=\csp{A}\ds\per{\csp{A}}=\csp{A}\ds\nsp{\adjoint{A}}. \end{equation*}$

🔗

Proof.

Theorem Theorem 1.3.7 provides the first equality and Theorem Theorem 1.3.6 gives the second.

🔗

Checkpoint 1.3.9.

Compute the orthogonal complement of the subspace $U\subset\complex{3}\text{.}$

\begin{equation*} U=\spn{\set{\colvector{1\\-1\\5}, \colvector{3\\1\\3} }} \end{equation*}

Solution

Form the matrix $A\text{,}$ whose columns are the two basis vectors given for $U$ and compute the null space $\nsp{\adjoint{A}}$ by row-reducing the matrix. (Theorem Theorem 1.3.6)

\begin{equation*} \adjoint{A}=\begin{bmatrix} 1 & -1 & 5\\ 3 & 1 & 3\end{bmatrix} \rref \begin{bmatrix} 1 & 0 & 2\\ 0 & 1 & -3\end{bmatrix} \end{equation*}

\begin{equation*} \per{U}=\nsp{\adjoint{A}}=\spn{\set{\colvector{-2\\3\\1}}} \end{equation*}

🔗

Checkpoint 1.3.10.

Compute the orthogonal complements of the two subspaces from Exercises Checkpoint 1.3.3 and Checkpoint 1.3.4. For the subspace of $\complex{5}$ verify that your first complement was not the orthogonal complement (or return to the exercise and find a complement that is not orthogonal).