Standard Normal and Z-Scores

Standardizing to a z-score puts any normal variable on a common scale with mean zero and unit variance.

The standard normal

The standard normal N(0, 1) has mean 0 and variance 1, with density φ(z) = (1/√(2π)) e^{−z²/2}. Its CDF is written Φ(z). Every normal distribution is an affine copy of this one, so tables and code for Φ suffice for all normal calculations.

The z-score

For X ~ N(μ, σ²), the z-score Z = (X − μ)/σ measures how many standard deviations X sits from its mean. A z-score of +2 means two σ above the mean. Standardization makes values from different distributions directly comparable.

Computing normal probabilities

P(X ≤ x) = Φ((x − μ)/σ). Tail probabilities, central intervals, and critical values all reduce to computations of Φ. Common critical values worth memorizing: Φ⁻¹(0.975) ≈ 1.96 and Φ⁻¹(0.995) ≈ 2.576, used for 95% and 99% two-sided intervals.

import math
def Phi(z):
    return 0.5*(1+math.erf(z/math.sqrt(2)))
print(round(Phi(1.96),4))  # 0.9750

Uses beyond probability

Z-scores are the standard way to flag outliers and to normalize features before machine learning, so that variables with large natural scales do not dominate distance calculations. Standardized residuals in regression use the same idea to judge whether a point deviates more than noise would explain.

One caution: standardizing does not make a non-normal variable normal. It only rescales, shifting the mean to zero and the standard deviation to one. If the underlying distribution is skewed or heavy-tailed, its z-scores are equally skewed, and normal-based tail probabilities will be wrong. Standardization is a change of units, not a change of shape.

Because Φ has no closed form, it is computed from the error function erf, which every numerical library provides.