From A First Course in Linear Algebra
Version 1.08
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/
This Section is a Draft, Subject to Changes
Needs Numerical Examples
Positive semi-definite matrices (and their cousins, positive definite matrices) are square matrices which in many ways behave like non-negative (respectively, positive) real numbers. Results given here are employed in the decompositions of Section SVD, Section SR and Section PD.
Definition PSM
Positive Semi-Definite Matrix
A square matrix
of size is positive
semi-definite if is
Hermitian and for all ,
.
For a definition of positive definite replace the inequality in the definition with a strict inequality, and exclude the zero vector from the vectors required to meet the condition. Similar variations allow definitions of negative definite and negative semi-definite. Our first theorem in this section gives us an easy way to build positive semi-definite matrices.
Theorem CPSM
Creating Positive Semi-Definite Matrices
Suppose that is
any matrix. Then
the matrices
and are positive
semi-definite matrices.
Proof We will give the proof for the first matrix, the proof for the second is entirely similar. First we check that is Hermitian,
so by Definition HM, the matrix is Hermitian. Second, for any ,
which is the second criteria in the definition of a postive semi-definite matrix (Definition PSM).
A statement very similar to the converse of this theorem is also true. Any positive semi-definite matrix can be realized as the product of a square matrix, , with its adjoint, . (See Exercise PSM.T20 after studying this entire section.) The matrices and will be important later when we define singular values (Section SVD).
Positive semi-definite matrices can also be characterized by their eigenvalues, without any mention of inner products. This next result further reinforces the notion that positive semi-definite matrices behave like non-negative real numbers.
Theorem EPSM
Eigenvalues of Positive Semi-definite Matrices
Suppose that is a
Hermitian matrix. Then
is positive semi-definite matrix if and only if whenever
is an
eigenvalue of ,
then .
Proof Notice first that since we are considering only Hermitian matrices in this theorem, it is always possible to compare eigenvalues with the real number zero, since eigenvalues of Hermitian matrices are all real numbers (Theorem HMRE). Let denote the size of .
() Let be an eigenvector of for . Then by Theorem PIP we know . So
By Theorem PIP, and by Definition PSM we have . With expressed as the product of these two quantities, we have .
() Suppose now that are the (not necessarily distinct) eigenvalues of the Hermitian matrix , each of which is non-negative. Let be a set of associated eigenvectors for these eigenvalues. Since a Hermitian matrix is normal (Definition HM, Definition NM), Theorem OBNM allows us to choose this set of eigenvectors to also be an orthonormal basis of . Choose any and let be the scalars guaranteed by the spanning property of the basis such that
Since we have presumed is Hermitian, we need only check the other defining property,
With non-negative values for each eigenvalue , , and each modulus squared, it should be clear that this sum is non-negative. Which is exactly what is required by Definition PSM to establish that is positive semi-definite.
As positive semi-definite matrices are defined to be Hermitian, they are then normal and subject to orthonormal diagonalization (Theorem OD). Now consider the interpretation of orthonormal diagonalization as a rotation to principal axes, a stretch by a diagonal matrix and a rotation back (Subsection OD.OD). For a positive semi-definite matrix, the diagonal matrix has diagonal entries that are the non-negative eigenvalues of the original positive semi-definite matrix. So the “stretching” along each axis is never a reflection.
T20 Suppose that
is a positive semi-definite matrix of size
. Prove that there
is a square matix
of size
such that .
Contributed by Robert Beezer