Monte Carlo Dropout

Keeping dropout active at prediction time and averaging many stochastic forward passes gives a cheap approximation to Bayesian uncertainty.

Dropout as inference

Dropout randomly zeros units during training as regularization. Gal and Ghahramani showed that leaving dropout on at test time and averaging repeated forward passes approximates variational inference in a deep Gaussian process. Each pass samples a different sub-network; the spread across passes estimates epistemic uncertainty.

The procedure

Kronos motion — monte carlo

Train the network with dropout as usual
At prediction, run T stochastic forward passes with dropout enabled
Report the mean as the prediction and the variance across passes as uncertainty

python

import numpy as np
# model in train mode so dropout stays active
preds = np.stack([model(x, training=True) for _ in range(T)])
mu = preds.mean(0)
epistemic = preds.var(0)

Why it is popular

It needs no architectural change beyond dropout layers and no separate training procedure, so any network already using dropout can produce uncertainty estimates at the cost of extra forward passes. This makes it the lowest-friction option for equipping an existing surrogate with uncertainty.

Known limitations

The quality of the uncertainty depends strongly on the dropout rate, which acts as a fixed prior and is rarely calibrated. MC dropout often underestimates uncertainty far from the data and does not always widen appropriately in extrapolation. It is best treated as a convenient approximation whose calibration must be checked, not as a rigorous posterior.

Comparison

Deep ensembles generally give better-calibrated uncertainty than MC dropout at higher training cost, while MC dropout is cheaper to train but needs many inference passes. Choosing between them is a trade of training cost against calibration quality; when decisions are safety-relevant, pair either with conformal prediction for coverage guarantees.