Acceptance Criteria and Tolerances
Deciding in advance how much error or discrepancy is acceptable turns V&V evidence into a defensible pass-or-fail decision.
How Good Is Good Enough
Verification and validation produce error estimates and discrepancies, but a decision needs a threshold: how much error is acceptable for the intended use. Acceptance criteria set that threshold in advance, tied to the decision the simulation supports. Without them, a discrepancy can always be argued as small enough or worried over as too large, and the evidence never resolves into a decision.
Setting the Criteria
- Derive the tolerance from the decision: how accurate must the quantity of interest be for the design margin to hold.
- Set it before the analysis, so the criterion is not adjusted to match the result.
- Account for the combined uncertainty, so the criterion is met with confidence, not just on central values.
Tolerances in Verification
In verification, the acceptance criterion is a numerical error target: the discretization error, from the Grid Convergence Index, must be below some fraction of the quantity of interest, small enough not to affect the decision. In regression testing, the tolerance distinguishes harmless round-off drift from a real regression. Setting these tolerances well requires knowing the accuracy the decision needs and the accuracy the numerics can deliver.
Tolerances in Validation
In validation, the criterion is a maximum acceptable discrepancy between simulation and experiment, judged relative to combined uncertainty. A model that agrees within the validation uncertainty passes for use in that regime; one that does not either needs improvement or must be used with added margin. Reporting the discrepancy against a pre-set criterion, rather than a bare pass or fail, preserves the information about how close the result was.
The discipline is to set criteria from the decision, in advance, and to document them. This prevents the two opposite abuses: relaxing the criterion after the fact to declare success, and tightening it to manufacture a problem. Acceptance criteria are what connect the technical evidence of V&V to the engineering decision it exists to support, and they belong in the documented record alongside the evidence itself.