The Curse of Dimensionality
As input dimensions grow, volume explodes and samples spread thin, defeating grid methods and straining every surrogate and UQ technique.
Volume explodes
The phrase, coined by Richard Bellman, names a family of problems that arise when the number of dimensions grows. The core cause is that the volume of a space grows exponentially with dimension. Ten points cover a line reasonably; the same density on a ten-dimensional cube needs ten billion points. Any method that tries to fill space uniformly becomes hopeless.
Everything becomes far and equidistant
- Nearest neighbors stop being near - distances to closest and farthest points converge
- Random samples all sit near the surface of a high-dimensional ball, not the center
- Most of a hypercube's volume lies in its corners
- Intuition built in two or three dimensions actively misleads
Consequences for surrogates
Grid-based quadrature and full-tensor designs are impossible past a handful of dimensions. Distance-based methods like RBF and Gaussian processes weaken as neighbors lose meaning. The number of polynomial chaos terms explodes combinatorially. The data needed to constrain any flexible model grows exponentially, so accuracy stalls.
What still works
Monte Carlo is the great exception: its convergence rate does not depend on dimension, which is why it dominates high-dimensional integration. Sparse grids, low-rank tensor formats, and additive or low-effective-dimension assumptions also help by exploiting structure the full space does not have.
Effective dimensionality
The saving grace is that most real problems have low effective dimension: only a few inputs, or a few combinations of inputs, actually drive the output. Sensitivity analysis, active subspaces, and dimension reduction find this hidden low-dimensional structure and let surrogates work in it rather than in the full space.
In fusion modeling
Kronos design spaces can nominally involve many parameters, but sensitivity analysis of the machines typically shows a small set of dominant inputs. Surrogates are built in that reduced effective space, and Monte Carlo handles the residual high-dimensional propagation, sidestepping the curse rather than fighting it head-on.