Linear Independence

A set of vectors is independent when none can be built from the others; this fixes the true dimension of a space.

Definition

Vectors v1, ..., vk are linearly independent if the only way to write the zero vector as a combination c1 v1 + ... + ck vk = 0 is with every coefficient ci equal to zero. If some nontrivial combination gives zero, the vectors are linearly dependent, meaning at least one is a combination of the others and therefore redundant.

Why it matters

Independence measures how much genuine information a set of vectors carries. A dependent set wastes directions: it spans the same space a smaller set could. The maximum size of an independent set inside a space equals that space's dimension, and any independent set of exactly that size is a basis.

How to test

Assemble the vectors as columns of a matrix A. They are independent if and only if the only solution to Ax = 0 is x = 0, equivalently if A has full column rank, equivalently if A^T A is invertible. For a square set, independence is equivalent to a nonzero determinant. In practice, computing the rank via a factorization is the robust test.

Geometric picture

Two vectors are dependent when they lie on a common line; three vectors are dependent when they lie in a common plane. Independence means the vectors reach out in truly distinct directions, so their span grows by one dimension per vector added.

import numpy as np
V = np.array([[1, 0, 1], [0, 1, 1], [1, 1, 2]]).T
print(np.linalg.matrix_rank(V))   # 2 < 3, so dependent

When choosing basis functions or diagnostic signals for a physics model, keeping them independent avoids ill-conditioned fits where redundant directions amplify measurement noise.