PCA for UQ

Principal component analysis compresses correlated inputs or outputs into a few uncorrelated components, simplifying uncertainty propagation.

What PCA does

Principal component analysis diagonalizes the covariance matrix of a dataset, producing orthogonal directions (principal components) ordered by the variance they explain. Projecting data onto the leading components yields a low-dimensional representation that retains most of the variability. In UQ it is used on both inputs and outputs.

Input-side use

Kronos motion — three outputs

Correlated uncertain inputs can be transformed to uncorrelated latent variables via PCA, reducing the effective input dimension and simplifying sampling. This is closely tied to the Karhunen-Loeve expansion, which is PCA applied to a random field's covariance.

Output-side use

High-dimensional simulation outputs (fields, spectra, time series) are often intrinsically low-dimensional. PCA compresses them so a surrogate predicts a few component scores instead of thousands of grid values. The reconstruction error from truncation must be tracked and added to the surrogate's own error budget.

Relationship to other methods

PCA vs KL: KL is the continuous form, PCA the discrete form on sampled data
PCA vs active subspaces: PCA ignores the output; active subspaces use output gradients to find directions that matter for a specific quantity
PCA vs autoencoders: autoencoders capture nonlinear structure PCA cannot

Cautions

PCA is variance-based and unsupervised, so a direction with small input variance can still strongly drive an output; do not discard components solely by input variance if the goal is prediction. Standardize variables before PCA when they have different units, and report the cumulative explained-variance ratio so readers can judge how much information the truncation discards.