Reproducible Simulation Pipelines
A reproducible pipeline lets anyone with the inputs regenerate a result exactly, through pinned environments, versioned data, and automated execution.
The reproducibility standard
A result is reproducible when someone else, given the inputs and the method, can obtain the same output. In computational science this is achievable and expected: the code, data, and environment are all digital and can be recorded exactly. A pipeline engineered for reproducibility makes regeneration routine rather than heroic.
The ingredients
- Version-controlled source code, pinned to an exact commit
- Versioned input data with content checksums
- A captured environment: pinned compilers, libraries, and settings
- An automated runner that executes the steps deterministically
- Recorded outputs with their provenance manifest
Environments
Software environments drift as libraries update, silently changing results. Reproducible pipelines pin the environment, through container images or lock files, so the same versions are used every time. This removes one of the commonest and most frustrating sources of non-reproducibility.
Determinism
Some computations, notably Monte-Carlo, use random numbers. Reproducibility here means fixing the random seed so a run repeats exactly, while separately assessing the statistical uncertainty by varying seeds. Parallel execution must also be made deterministic or its non-determinism documented and bounded.
Tiered reproducibility
Not every result reproduces bit-for-bit across machines; floating-point and parallel effects intervene. A practical standard distinguishes exact byte-level reproduction on the same environment from reproduction within a stated tolerance across environments. Both are legitimate as long as which one is claimed is stated clearly.
Kronos builds its analyses as reproducible pipelines so that published physics can be regenerated and independently checked.