Marginal and Conditional Distributions

From a joint distribution you extract marginals by summing out variables and conditionals by fixing them.

Two ways to reduce a joint

Given a joint distribution of X and Y, there are two natural one-variable summaries. The marginal ignores the other variable; the conditional fixes it. Both are derived from the same joint but answer different questions.

Marginals

The marginal of X sums (or integrates) the joint over all values of Y: p_X(x) = Σ_y p(x, y). It gives the distribution of X when you know nothing about Y. Marginalization is exactly how nuisance variables are removed from a model.

Conditionals

The conditional of X given Y = y renormalizes a slice of the joint: p(x | y) = p(x, y) / p_Y(y). It describes X once Y is known to be y. Conditioning is how observations update a joint model.

The chain rule and Bayes

The joint always factors as p(x, y) = p(x | y) p_Y(y) = p(y | x) p_X(x). Equating these two factorizations gives Bayes theorem for densities, p(x | y) = p(y | x) p_X(x) / p_Y(y). This is the engine of Bayesian inference over continuous parameters.

Marginal versus conditional independence

Two variables can be marginally dependent but conditionally independent given a third, or the reverse. Simpson's paradox — a trend that reverses when data are pooled across groups — is the vivid warning that marginal and conditional relationships can point in opposite directions.