Floating-Point and Reproducibility
Finite-precision arithmetic obeys its own rules; knowing them explains why numerical results drift and how to keep them stable.
Real Numbers on a Finite Machine
Computers represent most real numbers approximately, using a fixed number of bits split between a significand and an exponent, following the IEEE 754 standard. The gap between representable numbers grows with magnitude, so every operation may round. These rounding errors are tiny individually but can accumulate or amplify.
Rules That Surprise Newcomers
- Addition is not associative: the order of a long sum changes its rounded value.
- Subtracting nearly equal numbers loses significant digits, called cancellation.
- Equality comparison is fragile; test within a tolerance instead.
- Some decimal fractions, like 0.1, have no exact binary representation.
Conditioning and Stability
Two separate ideas govern accuracy. Conditioning is a property of the problem: how much the true answer changes when inputs are perturbed. Stability is a property of the algorithm: how much it amplifies rounding error. A well-conditioned problem solved by an unstable algorithm still gives poor answers, and a good algorithm cannot rescue an ill-conditioned problem.
Working With It, Not Against It
Practitioners reduce error by summing from small to large, using compensated summation for long sums, reformulating expressions to avoid cancellation, and choosing stable algorithms. Where more precision is genuinely needed, higher-precision types exist, at a cost in speed and memory.
Why It Matters for Reproducibility
Because the last digits depend on operation order, they depend on hardware, threads, and compilers. Reproducing a result therefore means agreeing within a tolerance chosen to exceed expected rounding differences, not demanding identical bits. Kronos runbooks state such tolerances explicitly so a reader can tell a real discrepancy from arithmetic noise.