Model Calibration
Tuning uncertain model parameters to match data, a legitimate step that becomes self-deception when confused with validation.
Fitting the Free Parameters
Many models contain parameters that are not known from first principles: a turbulence closure coefficient, an unmeasured rate, an effective transport factor. Calibration adjusts these parameters so the model reproduces observed data. Done carefully, it is a legitimate and necessary step that turns a model with free knobs into a predictive tool.
The Central Danger
Calibration is not validation. A model tuned to match a data set will, unsurprisingly, match that data set. Reporting that agreement as validation is circular: the agreement was engineered. The only honest validation test is against data the model was not calibrated on, which is why calibration and validation data must be kept strictly separate.
Overfitting
- Too many free parameters can fit the calibration data while predicting poorly elsewhere.
- A parameter that absorbs the effect of missing physics will fail when the regime changes.
- Calibrated parameters should stay within physically plausible ranges; a good fit at an absurd value signals a model defect, not a discovery.
Bayesian Calibration
A principled framework treats the parameters as uncertain and updates their probability distributions using the data, yielding not point estimates but posteriors that carry uncertainty forward. This makes explicit how much the data actually constrains each parameter, and it exposes parameters the data cannot pin down at all, which point estimates hide.
The discipline that keeps calibration honest is bookkeeping: state which parameters were calibrated, on what data, over what ranges, and hold out independent data for validation. A calibrated model reported as validated on its own calibration data is one of the most common and most damaging errors in computational science, because it produces confident predictions with no real support.