Positive-Definite Matrices

Symmetric matrices whose quadratic form is always positive; they behave like positive numbers among matrices.

Definition

A symmetric matrix A is positive definite if the quadratic form x^T A x is strictly positive for every nonzero vector x. If it is merely nonnegative, A is positive semidefinite. These matrices are the matrix analogue of positive numbers, and they carry the strongest possible good behavior for computation and optimization.

Equivalent tests

• all eigenvalues are strictly positive
• a Cholesky factorization A = L L^T exists
• all leading principal minors are positive (Sylvester's criterion)
• x^T A x > 0 for every nonzero x

Any of these conditions implies the others, so in practice one checks whichever is cheapest: attempting Cholesky is a fast and reliable test.

Why they matter in optimization

A twice-differentiable function has a strict local minimum where its gradient vanishes and its Hessian is positive definite. Convex quadratic problems with a positive-definite Hessian have a unique global minimum reachable by efficient methods. Positive definiteness is thus the condition that guarantees an optimization landscape curves upward in every direction.

Sources of these matrices

Products of the form A^T A are always positive semidefinite, and positive definite when A has independent columns; this is why least-squares systems are well behaved. Covariance matrices, stiffness matrices, and Gram matrices of independent vectors are all positive definite, tying the concept to statistics, mechanics, and geometry.

import numpy as np
def is_pos_def(A):
    try:
        np.linalg.cholesky(A); return True
    except np.linalg.LinAlgError:
        return False
print(is_pos_def(np.array([[2.0, -1.0], [-1.0, 2.0]])))   # True

Discretized energy operators in physics are typically symmetric positive definite, guaranteeing stable solves and physically meaningful, real, positive mode frequencies.