Golden Master Testing
Record the output of a trusted run as a reference, then flag any future change from it; a blunt but powerful safety net.
Recording a Reference
Golden master testing, also called characterization or snapshot testing, records the complete output of a code run when the code is believed correct, then compares future runs against that stored reference. Any deviation is flagged. It does not check whether the output is right in an absolute sense, only whether it has changed, which makes it a blunt but effective way to detect unintended effects of a change.
When It Shines
- Legacy code with no tests, where writing targeted tests is hard but capturing current behavior is easy.
- Complex outputs, such as full field dumps, that are hard to specify but easy to compare.
- Refactoring, where the goal is explicitly to change the code without changing the output.
The Central Weakness
A golden master only certifies that behavior is unchanged, not that it is correct. If the recorded reference was wrong, the test faithfully preserves the bug and flags any fix as a failure. The master must therefore be established from a run that is genuinely trusted, ideally one that has passed verification tests, and it must be updated deliberately when a change is intended, with the update reviewed rather than rubber-stamped.
Managing the Reference
The stored reference is data that must be version-controlled alongside the code, so that the correct master for each code version is unambiguous. Comparisons use a tolerance for floating-point fields. When a change legitimately alters output, updating the master is a reviewed decision: the reviewer confirms the new output is correct, not merely different, before it becomes the new reference.
Golden master tests complement rather than replace verification. Verification establishes that the reference output was correct in the first place; the golden master then guards that correct output against silent drift. Used together, they give both an absolute correctness check and a continuous change alarm.