Verification and Reproducibility
A result you cannot reproduce is a rumor; verification and reproducibility turn computation into evidence.
The discipline
Reproducibility means that the same inputs, run again, yield the same outputs — by anyone, on a documented environment. Verification means checking that a computation solves the problem it claims to solve. Together they are what separate an engineering result from a lucky screenshot.
What reproducibility requires
- Fixed, recorded inputs and parameters.
- A pinned software environment: versions of every library and tool.
- Controlled randomness via recorded seeds.
- A recorded procedure that a stranger could follow.
Verification in layers
Verification proceeds from small to large: unit tests on individual functions, method-of-manufactured-solutions checks on solvers, convergence studies as the mesh or timestep refines, and comparison against analytic cases with known answers. Each layer catches a different class of error.
Tiers of reproducibility
Not all reproduction is bit-for-bit. A useful distinction is Tier-1 byte-identical reproduction, achievable when the environment is fully pinned, versus Tier-2 reproduction within stated tolerances, which accounts for hardware and library differences. Declaring which tier a result meets is itself part of honesty.
Why it is non-negotiable
Fusion design decisions rest on simulation. If a key result cannot be regenerated on demand, it cannot be defended, corrected, or built upon. Treating reproducibility as a first-class deliverable — not an afterthought — is what lets a program stand behind numbers like a computed Q of 3.424 for the breeder Hyperion.