GMRES and Krylov Solvers
Krylov subspace methods solve large sparse linear systems by building an optimal solution from repeated matrix-vector products.
Solutions from a growing subspace
Krylov subspace methods solve A*x = b using only matrix-vector products with A. Starting from the residual, they build the Krylov subspace spanned by b, A*b, A^2*b, and so on, and seek the best approximate solution within it. Each iteration expands the subspace by one matrix-vector product and improves the approximation, so the methods are matrix-free-friendly and scale to enormous sparse systems.
Different Krylov methods differ in what optimality they enforce and what matrix properties they require. This choice determines convergence behavior, memory use, and cost per iteration.
GMRES and its cousins
The generalized minimal residual method (GMRES) minimizes the residual norm over the Krylov subspace and works for general nonsymmetric matrices. Its drawback is growing memory and cost per iteration, since it must store the whole subspace; restarted GMRES(m) caps this by discarding history every m steps, trading guaranteed convergence for bounded cost. For symmetric systems, MINRES and the conjugate gradient method are cheaper because short recurrences avoid storing the full basis. BiCGStab offers a short-recurrence option for nonsymmetric problems.
Convergence and preconditioning
Krylov convergence depends on the distribution of the matrix's eigenvalues (or singular values): clustered spectra converge fast, spread-out or ill-conditioned spectra converge slowly. Preconditioning transforms the system to cluster the spectrum and is almost always necessary for practical performance; the preconditioner often matters more than the Krylov method chosen.
- Solution built from repeated matrix-vector products
- GMRES minimizes residual for nonsymmetric matrices
- Restarting bounds GMRES memory at a convergence cost
- Preconditioning to cluster the spectrum is decisive
GMRES with a strong preconditioner is the standard inner solver in implicit and Newton-Krylov fusion codes, handling the large nonsymmetric systems that arise from coupled transport and field equations.