Reproducible Research
A result you cannot regenerate from its inputs is a claim, not a finding; reproducibility is what makes computational science checkable.
A Ladder of Terms
The community distinguishes degrees. Reproducibility: the same data and same code yield the same result. Replicability: new data or an independent implementation yield a consistent result. Repeatability: the same team gets the same result on a rerun. The weakest rung, computational reproducibility, is a minimum bar, not the finish line.
What It Takes
- The exact code, at a tagged version under version control.
- The exact input data, archived and checksummed.
- The computational environment: compiler, libraries, and their versions.
- The commands and parameters that produced each figure.
- A record of any randomness, including seeds.
Why Results Fail to Reproduce
Common causes include unrecorded parameter tweaks, dependence on a library version that later changed, uncontrolled randomness, manual steps that were never scripted, and floating-point differences across hardware. Each is avoidable with discipline, and each is nearly impossible to diagnose after the fact if the provenance was not captured at the time.
Reproducible Is Not Correct
A reproducible result can still be wrong; reproducibility guarantees only that the same inputs give the same output, not that the output is right. It is a precondition for scrutiny, not a substitute for it. Validation against reality remains a separate obligation.
At Kronos
Frozen physics results are tied to a specific code version, checksummed inputs, and a recorded environment, with published deposits carrying the files needed to regenerate key numbers. The aim is that an independent reader can rerun the computation and get the same value, which is the floor beneath any claim that the simulation supports a design.