AI & Foundations

Floating-Point Arithmetic

Computers represent real numbers with finite precision, so arithmetic carries small, structured errors that engineers must respect.

Finite representation

A computer cannot store most real numbers exactly. Floating point represents a number as a sign, a fraction, and an exponent, giving a fixed number of significant digits. Double precision provides roughly 15–16 decimal digits; single precision about 7.

Machine epsilon

Kronos motion — operating point

The smallest gap between representable numbers near 1.0 is called machine epsilon, about 2.2 × 10⁻¹⁶ for double precision. Any arithmetic result is rounded to the nearest representable value, so each operation can introduce error at this scale.

Where it bites

Subtracting two nearly equal numbers can cancel most significant digits.
Summing many small numbers can lose the small ones entirely.
Comparing floats for exact equality is almost always a bug.

A concrete surprise

python

# 0.1 + 0.2 is not exactly 0.3
print(0.1 + 0.2 == 0.3)   # False
print(0.1 + 0.2)          # 0.30000000000000004

# Compare with a tolerance instead
import math
print(math.isclose(0.1 + 0.2, 0.3))  # True

Living with it

Well-written numerical code is arranged to minimize catastrophic cancellation, sums are ordered or compensated for accuracy, and comparisons use tolerances. Understanding floating point is also why bit-for-bit reproducibility depends on fixing hardware, compiler, and library versions: the same math in a different order can give a slightly different last digit.