Randomized Numerical Linear Algebra
Using random projections to compute approximate factorizations of huge matrices far faster than deterministic methods.
Randomness as a computational tool
For very large matrices, classical factorizations are too expensive, especially when only a low-rank approximation is needed. Randomized algorithms use random sampling to capture the dominant action of a matrix cheaply. The surprising and rigorous result is that a small number of random projections almost always suffices to reveal a matrix's leading subspace with high accuracy, with failure probability that is negligibly small.
The randomized range finder
The core operation approximates the column space of a matrix A. Multiply A by a tall random matrix (Gaussian or structured), producing a thin matrix whose columns are random combinations of A's columns. Orthonormalize these to get a basis Q that, with high probability, captures the dominant range of A. The matrix is then compressed by projecting onto Q, and a small factorization is computed on the compressed result and mapped back.
- Randomized SVD: cheap approximate leading singular values and vectors
- Oversampling: take a few extra random vectors to boost accuracy and reliability
- Power iterations: a few applications of A times A transposed sharpen the captured subspace when singular values decay slowly
- Single-pass variants: build the approximation while reading the matrix only once, for streaming or out-of-core data
Why it is fast
The expensive step is reduced to a few matrix-matrix multiplications, which are highly parallel and cache-efficient (BLAS level 3), followed by a small dense factorization. For a matrix with rapidly decaying singular values, the randomized SVD matches the accuracy of the deterministic method at a fraction of the cost, and it parallelizes far better.
Use and reliability
Randomized methods power large-scale data analysis, model reduction, and machine learning, wherever a huge matrix must be compressed to its dominant structure. Their error bounds are probabilistic but extremely tight, and oversampling makes failure practically impossible, so they are trusted in production numerical software for exactly the low-rank problems that arise when reducing high-dimensional simulation data.