Designing a Validation Experiment
A validation experiment is built to test a model, which means measuring the inputs the model needs as carefully as the outputs.
Built to Test a Model
A validation experiment differs in purpose from a discovery experiment. Its job is to test a model as cleanly as possible, which imposes requirements that discovery experiments often ignore. Above all, the inputs the model needs, boundary conditions, material properties, and initial state, must be measured, not assumed, because a discrepancy between model and data is only diagnostic if the inputs are known.
Requirements of a Validation Experiment
- Well-characterized inputs: every quantity the model needs is measured with quantified uncertainty.
- Measured outputs with uncertainty: the quantities of interest, with their measurement error.
- Sufficient instrumentation: enough diagnostics to constrain the comparison, not just a single output.
- Repeatability: repeated runs to quantify experimental variability.
The Input Problem
The most common failure in validation is unmeasured inputs. If the model needs a boundary condition or material property that was not measured, the modeler must assume it, and any discrepancy could be blamed on the assumption rather than on the model. A well-designed validation experiment closes this loophole by measuring everything the model consumes, so the comparison isolates the model's physics.
Matching to the Hierarchy
Validation experiments are designed to fit the validation hierarchy. Unit experiments isolate a single phenomenon under controlled conditions, so the model of that phenomenon is tested cleanly. Integrated experiments combine phenomena to test their coupling. Designing the right experiment for the right level of the hierarchy is what lets confidence be built from the bottom up, with each level's discrepancy attributable to the physics that level introduced.
Because good validation experiments are expensive, sensitivity analysis is used to design them well: the experiment should be most informative about the inputs and phenomena the design prediction is most sensitive to, so the validation effort tightens the predictions that matter most.