Machine Epsilon and Rounding

Machine epsilon is the smallest gap between 1 and the next representable number, and it sets the relative accuracy of every floating-point operation.

The unit of relative error

Machine epsilon, often written eps or u, is the distance from 1.0 to the next larger floating-point number. For IEEE double precision it is 2^-52, about 2.22 x 10^-16. It measures the best possible relative precision: any real number x rounded to floating point satisfies fl(x) = x(1 + delta) with |delta| <= eps/2 under round-to-nearest.

Machine epsilon is not the smallest representable number. The smallest positive normalized double is about 2.2 x 10^-308, and subnormals reach far smaller. Epsilon is about spacing near 1, which scales with magnitude: near a number of size 2^k, the spacing between representable values is 2^k times eps.

Rounding modes

IEEE 754 defines several rounding modes. The default is round to nearest, ties to even, which minimizes bias by rounding halfway cases to the value with an even last bit. Other modes round toward zero, toward positive infinity, or toward negative infinity, which are useful for interval arithmetic and error bounds.

The standard model of arithmetic

Each basic operation (add, subtract, multiply, divide, square root) returns the exactly rounded result. Formally, fl(a op b) = (a op b)(1 + delta) with |delta| <= eps/2. This single assumption underlies almost all rounding-error analysis: it lets analysts bound how errors grow through a sequence of operations.

eps = 1.0
while 1.0 + eps/2.0 != 1.0:
    eps /= 2.0
print(eps)   # ~2.22e-16, machine epsilon for float64

Because each operation carries a relative error near eps, the central question in numerical analysis is not whether errors appear, but whether an algorithm keeps them from growing catastrophically.