A Second Course in Linear Algebra

• Preface
• Dedication and Acknowledgements

Canonical Forms

• Nilpotent Linear Transformations
• Invariant Subspaces
• Jordan Canonical Form

Matrix Decompositions

• Rank One Decomposition
• Triangular Decomposition
• Singular Value Decomposition
• Square Roots
• Polar Decomposition

Topics

• Fields
• Trace
• Hadamard Product
• Vandermonde Matrix
• Positive Semi-Definite Matrices

Applications

• Curve Fitting
• Sharing a Secret

Archetypes

•   ABCDEFGHIJKLM
•   NOPQRSTUVWX

Reference

• Notation
• Definitions
• Theorems
• Examples
• Sage
• Proof Techniques
• GFDL License

Subsection

{\sc\large This Section is a Draft, Subject to Changes}\\ {\sc\large Needs Numerical Examples}\\ {\sc\large Inner product is “reversed” from prior material, see changelog explanation}

\bigskip Positive semi-definite matrices (and their cousins, positive definite matrices) are square matrices which in many ways behave like non-negative (respectively, positive) real numbers. Results given here are employed in the decompositions of Section SVD, Section SR and Section PD. \subsect{PSM}{Positive Semi-Definite Matrices}

Definition PSM Positive Semi-Definite Matrix

A square matrix $A$ of size $n$ is positive semi-definite if $A$ is Hermitian and for all $\vect{x}\in\complex{n}$, $\innerproduct{A\vect{x}}{\vect{x}}\geq 0$.

$\square$

For a definition of positive definite replace the inequality in the definition with a strict inequality, and exclude the zero vector from the vectors $\vect{x}$ required to meet the condition. Similar variations allow definitions of negative definite and negative semi-definite. Our first theorem in this section gives us an easy way to build positive semi-definite matrices.

Theorem CPSM Creating Positive Semi-Definite Matrices

Suppose that $A$ is any $m\times n$ matrix. Then the matrices $\adjoint{A}A$ and $A\adjoint{A}$ are positive semi-definite matrices.

Proof

A statement very similar to the converse of this theorem is also true. Any positive semi-definite matrix can be realized as the product of a square matrix, $B$, with its adjoint, $\adjoint{B}$. (See Exercise PSM.T20 after studying this entire section.) The matrices $\adjoint{A}A$ and $A\adjoint{A}$ will be important later when we define singular values (Section SVD).

Positive semi-definite matrices can also be characterized by their eigenvalues, without any mention of inner products. This next result further reinforces the notion that positive semi-definite matrices behave like non-negative real numbers.

Theorem EPSM Eigenvalues of Positive Semi-definite Matrices

Suppose that $A$ is a Hermitian matrix. Then $A$ is positive semi-definite matrix if and only if whenever $\lambda$ is an eigenvalue of $A$, then $\lambda\geq 0$.

Proof

As positive semi-definite matrices are defined to be Hermitian, they are then normal and subject to orthonormal diagonalization (Theorem OD). Now consider the interpretation of orthonormal diagonalization as a rotation to principal axes, a stretch by a diagonal matrix and a rotation back (Subsection OD.OD). For a positive semi-definite matrix, the diagonal matrix has diagonal entries that are the non-negative eigenvalues of the original positive semi-definite matrix. So the “stretching” along each axis is never a reflection.