Diffusion Models

Diffusion models generate data by learning to reverse a gradual noising process, denoising random noise step by step into a sample.

Forward and reverse processes

A diffusion model defines a forward process that slowly adds Gaussian noise to data over many steps until it becomes pure noise. It then trains a network to run the reverse process: given a noisy sample and the current step, predict how to remove a little noise. Generation starts from random noise and applies the learned reverse steps repeatedly, gradually turning noise into a coherent sample. The forward process is fixed; only the reverse denoiser is learned.

The training objective

Kronos motion — data assimilation

Rather than predict the clean image directly, the network usually predicts the noise that was added at a given step. Training samples a data point, a random step, and a noise vector, forms the corresponding noisy input, and minimizes the error between the predicted and actual noise. This simple regression objective is stable to train, a major practical advantage over the adversarial game of GANs.

python

# diffusion training step (sketch)
# x0 ~ data ; t ~ uniform(1..T) ; eps ~ N(0, I)
# x_t = sqrt(a_bar_t) * x0 + sqrt(1 - a_bar_t) * eps
# loss = || eps - model(x_t, t) ||^2

Sampling and speed

Generation traditionally required hundreds or thousands of denoising steps, making diffusion slower than a single-pass GAN. Faster samplers (DDIM, higher-order solvers) and distillation cut this to tens of steps or fewer while preserving quality. Latent diffusion runs the process in a compressed latent space rather than pixel space, greatly reducing cost and enabling high-resolution generation.

Conditioning and guidance

Diffusion models are steered by conditioning the denoiser on extra inputs, most famously text prompts via cross-attention to text embeddings. Classifier-free guidance amplifies the influence of the condition by mixing conditional and unconditional predictions, trading diversity for prompt adherence. This machinery underlies modern text-to-image systems and extends to audio, video, and 3D.

Why they took over

Diffusion models combine high sample quality, broad diversity, and stable training, avoiding the mode collapse and delicate balancing of GANs. Their main drawback is slower sampling, which ongoing work steadily reduces. They now lead general-purpose image, audio, and video generation, and the denoising framework is being adapted to scientific problems such as generating candidate molecular structures and field configurations that satisfy learned constraints.

Learn to reverse a fixed noising process.
Predict the added noise with a simple regression loss.
Stable training, but many sampling steps unless accelerated.
Conditioning and guidance enable text-to-image generation.