Normalizing Flows

Normalizing flows build generative models from invertible transformations, giving exact likelihoods where VAEs and GANs give only approximations.

Exact density by change of variables

A normalizing flow transforms a simple base distribution, usually a standard Gaussian, into a complex data distribution through a sequence of invertible functions. Because each transformation is invertible and differentiable, the change-of-variables formula gives the exact probability density of any data point. Unlike VAEs, which optimize a lower bound, and GANs, which have no explicit density, flows compute the true likelihood, making them useful when calibrated probabilities matter.

The change-of-variables formula

Kronos motion — radial build

If z is a base sample and x = f(z) with f invertible, the density of x is the base density of f-inverse(x) multiplied by the absolute value of the determinant of the inverse transformation's Jacobian. Composing many transformations multiplies their Jacobian determinants. The engineering challenge is designing transformations that are expressive, easily invertible, and have a Jacobian determinant that is cheap to compute.

Coupling layers

The trick used by RealNVP and Glow is the coupling layer. Split the input in half; leave one half unchanged and transform the other half using a function of the untouched half. This makes the Jacobian triangular, so its determinant is just the product of diagonal entries, cheap to compute, and inversion is straightforward. Stacking many coupling layers, alternating which half is transformed, builds an expressive yet tractable flow.

python

# affine coupling layer (forward)
# x1, x2 = split(x)
# y1 = x1
# y2 = x2 * exp(s(x1)) + t(x1)   # s,t are neural nets
# log|det J| = sum(s(x1))

Trade-offs

Exact likelihood and exact, efficient sampling in both directions are the strengths. The cost is architectural constraint: every layer must be invertible with a tractable Jacobian, which limits flexibility and often requires more parameters than a comparable VAE or GAN. Sample quality has historically trailed GANs and diffusion on images, though flows remain competitive in lower dimensions and where density estimation is the goal.

Where flows shine

Because they provide exact densities, flows are strong for density estimation, likelihood-based anomaly detection, and probabilistic inference. They are used in physics and chemistry to model distributions such as molecular conformations and lattice-field configurations, where a valid probability density and unbiased sampling are more important than photorealistic output. Continuous-time flows connect the idea to differential equations.

Invertible transforms map a simple base to complex data.
Change of variables yields exact likelihoods.
Coupling layers keep the Jacobian tractable.
Best where calibrated density estimation matters.