Section PSM  Positive Semi-definite Matrices

From A First Course in Linear Algebra
Version 2.00
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/

This Section is a Draft, Subject to Changes
Needs Numerical Examples

Positive semi-definite matrices (and their cousins, positive definite matrices) are square matrices which in many ways behave like non-negative (respectively, positive) real numbers. Results given here are employed in the decompositions of Section SVD, Section SR and Section PD.

Subsection PSM: Positive Semi-Definite Matrices

Definition PSM
Positive Semi-Definite Matrix
A square matrix A of size n is positive semi-definite if A is Hermitian and for all x n, Ax,x 0.

For a definition of positive definite replace the inequality in the definition with a strict inequality, and exclude the zero vector from the vectors x required to meet the condition. Similar variations allow definitions of negative definite and negative semi-definite. Our first theorem in this section gives us an easy way to build positive semi-definite matrices.

Theorem CPSM
Creating Positive Semi-Definite Matrices
Suppose that A is any m × n matrix. Then the matrices AA and AA are positive semi-definite matrices.

Proof   We will give the proof for the first matrix, the proof for the second is entirely similar. First we check that AA is Hermitian,

AA = AA  Theorem MMAD = AA  Theorem AA

so by Definition HM, the matrix AA is Hermitian. Second, for any x n,

AAx,x = Ax, Ax  Theorem AIP = Ax,Ax  Theorem AA 0  Theorem PIP

which is the second criteria in the definition of a postive semi-definite matrix (Definition PSM).

A statement very similar to the converse of this theorem is also true. Any positive semi-definite matrix can be realized as the product of a square matrix, B, with its adjoint, B. (See Exercise PSM.T20 after studying this entire section.) The matrices AA and AA will be important later when we define singular values (Section SVD).

Positive semi-definite matrices can also be characterized by their eigenvalues, without any mention of inner products. This next result further reinforces the notion that positive semi-definite matrices behave like non-negative real numbers.

Theorem EPSM
Eigenvalues of Positive Semi-definite Matrices
Suppose that A is a Hermitian matrix. Then A is positive semi-definite matrix if and only if whenever λ is an eigenvalue of A, then λ 0.

Proof   Notice first that since we are considering only Hermitian matrices in this theorem, it is always possible to compare eigenvalues with the real number zero, since eigenvalues of Hermitian matrices are all real numbers (Theorem HMRE). Let n denote the size of A.

( ) Let x0 be an eigenvector of A for λ. Then by Theorem PIP we know x,x0. So

λ = 1 x,xλ x,x  Property MICN = 1 x,x λx,x  Theorem IPSM = 1 x,x Ax,x  Definition EEM

By Theorem PIP, x,x > 0 and by Definition PSM we have Ax,x 0. With λ expressed as the product of these two quantities, we have λ 0.

( ) Suppose now that λ1,λ2,λ3,,λn are the (not necessarily distinct) eigenvalues of the Hermitian matrix A, each of which is non-negative. Let B = x1,x2,x3,,xn be a set of associated eigenvectors for these eigenvalues. Since a Hermitian matrix is normal (Definition HM, Definition NM), Theorem OBNM allows us to choose this set of eigenvectors to also be an orthonormal basis of n. Choose any x n and let a1,a2,a3,,an be the scalars guaranteed by the spanning property of the basis B such that

x = a1x1 + a2x2 + a3x3 + + anxn = i=1na ixi

Since we have presumed A is Hermitian, we need only check the other defining property,

Ax,x = A i=1na ixi, j=1na jxj  Definition TSVS = i=1nAa ixi, j=1na jxj  Theorem MMDAA = i=1na iAxi, j=1na jxj  Theorem MMSMM = i=1na iλixi, j=1na jxj  Definition EEM = i=1n j=1n a iλixi,ajxj  Theorem IPVA = i=1n j=1na iλiaj¯ xi,xj  Theorem IPSM = i=1na iλiai¯ xi,xi + i=1n j=1 ji na iλiaj¯ xi,xj  Property CACN = i=1na iλiai¯(1) + i=1n j=1 ji na iλiaj¯(0)  Definition ONS = i=1na iλiai¯ = i=1nλ i ai 2  Definition MCN

With non-negative values for each eigenvalue λi, 1 i n, and each modulus squared, it should be clear that this sum is non-negative. Which is exactly what is required by Definition PSM to establish that A is positive semi-definite.

As positive semi-definite matrices are defined to be Hermitian, they are then normal and subject to orthonormal diagonalization (Theorem OD). Now consider the interpretation of orthonormal diagonalization as a rotation to principal axes, a stretch by a diagonal matrix and a rotation back (Subsection OD.OD). For a positive semi-definite matrix, the diagonal matrix has diagonal entries that are the non-negative eigenvalues of the original positive semi-definite matrix. So the “stretching” along each axis is never a reflection.

Subsection EXC: Exercises

T20 Suppose that A is a positive semi-definite matrix of size n. Prove that there is a square matix B of size n such that A = BB.  
Contributed by Robert Beezer