Uncertainty Quantification in ML
Uncertainty quantification separates noise in the data from ignorance in the model, so predictions come with honest, actionable error bars.
Why point predictions are not enough
A single predicted number hides how much the model actually knows. For scientific and engineering decisions, the reliability of a prediction matters as much as its value. Uncertainty quantification (UQ) attaches a calibrated measure of confidence to each prediction, distinguishing regions the model understands from regions where it is guessing.
Two kinds of uncertainty
Aleatoric uncertainty is irreducible noise inherent in the data, measurement error or genuine randomness, that more data cannot remove. Epistemic uncertainty is the model's own ignorance, which shrinks as more data are collected. Separating the two is important: epistemic uncertainty tells you where new data would help, while aleatoric uncertainty tells you the floor on achievable accuracy.
Methods
- Bayesian neural networks: place distributions over weights and infer them
- Deep ensembles: train several models and use their spread as uncertainty
- Monte Carlo dropout: sample dropout masks at prediction time
- Gaussian processes: give exact predictive variance in closed form
- Conformal prediction: wrap any model in distribution-free coverage guarantees
Calibration
An uncertainty estimate is only useful if it is calibrated: when the model says it is ninety percent confident, it should be right about ninety percent of the time. Calibration is checked on held-out data with reliability diagrams and scores such as the expected calibration error. An overconfident model is often more dangerous than an inaccurate but honest one.
Why it matters for science
Scientific decisions carry consequences, so a surrogate used in design must report where it can be trusted. Uncertainty flags extrapolation beyond the training regime, guides where to run expensive high-fidelity simulations next, and keeps a fast learned model from being mistaken for verified physics. UQ is what makes a surrogate safe to use rather than merely fast.