Reproducible Pipelines

A result is only ours to claim if someone else can rerun the pipeline that produced it and get the same answer; we build the pipelines to make that possible.

From inputs to figures, on someone else's machine

A reproducible pipeline is the full chain from raw inputs to the figures in a paper, packaged so that an independent reader can run it end to end and recover the published result. Kronos treats this as the acceptance test for a claim: if the pipeline does not reproduce, the number is not yet evidence. This is the practical core of designing in the open — the theory made runnable.

What goes into the deposit

The code, under an open license, so it can legally be run; see code under Apache.
The input data and parameters that define the design point.
A pinned environment so the code runs the same way elsewhere; see environment pinning.
A runbook that walks a reader from download to reproduced figure; see the cold-reproduction runbook.
A verification step that reports whether the reproduction matched; see the verify function.

A worked idea of what a runbook does

python

# reproduce a published design point, end to end
from kronos_repro import load_inputs, run_sweep, verify

inputs = load_inputs('breeder/design_point.yaml')   # exact published parameters
result = run_sweep(inputs)                            # rerun the search

# Q_sci, fusion power, plasma current recovered from the rerun
print(result.Q_sci, result.P_fus_MW, result.Ip_MA)   # -> 3.424 88.7 9.86

verify(result, reference='breeder/reference.json')    # tiered check, must pass

Two tiers of success

Not every result can match to the last bit; floating-point and hardware differences make that unrealistic across environments. Kronos therefore checks reproduction at two tiers — exact byte match where achievable, numerical tolerance otherwise — described under two-tier reproduction. The tier for each output is fixed in advance, so success has a defined meaning rather than a negotiated one.

What a working pipeline forces on us

Building for external reproduction changes how the work is done, not just how it is shipped. A parameter cannot be a number in someone's notebook; it must live in a versioned input file. An analysis cannot depend on a step performed by hand; it must be scripted end to end. A figure cannot be touched up after the fact; it must fall out of the code. That discipline is demanding, and it is exactly what makes a design-and-simulation result trustworthy: there is no manual stage where an unrecorded choice could quietly shape the answer.

The pipeline is the claim. Everything else is commentary on it.