Rank

The number of independent directions a matrix can reach, equal to the dimension of its column space.

Definition

The rank of a matrix is the maximum number of linearly independent columns, which always equals the maximum number of linearly independent rows. Equivalently, it is the dimension of the column space (the range of the transformation) and also the dimension of the row space. A matrix is full rank when its rank equals the smaller of its two dimensions.

Rank and solvability

Rank governs the behavior of linear systems. A square matrix is invertible exactly when it has full rank. For Ax = b, a solution exists when b lies in the column space, which happens precisely when the rank of A equals the rank of the augmented matrix [A | b]. When the rank is less than the number of unknowns, solutions, if any, form an infinite family.

The rank-nullity theorem

For an m-by-n matrix, rank plus nullity equals n, the number of columns. The nullity is the dimension of the null space, the set of vectors sent to zero. This conservation law says that every dimension of the input either survives into the output (contributing to rank) or is annihilated (contributing to nullity).

Numerical rank

In floating point, exact rank is fragile: rounding turns exact zeros into tiny nonzeros. The reliable measure is the numerical rank, the number of singular values above a tolerance. The singular value decomposition is the standard tool for estimating rank, far more robust than counting nonzero pivots or evaluating a determinant.

import numpy as np
A = np.array([[1, 2, 3], [2, 4, 6], [1, 0, 1]])
print(np.linalg.matrix_rank(A))   # 2, rows 1 and 2 are dependent

Low numerical rank in a data or operator matrix signals redundancy that low-rank approximation can compress, a fact exploited in reduced-order models of complex physical systems.