Conjugate Priors

A conjugate prior keeps the posterior in the same family as the prior, making Bayesian updates a closed-form calculation.

The idea

A prior is conjugate to a likelihood when the resulting posterior belongs to the same distributional family as the prior. Then updating means adjusting the family's parameters rather than integrating — the posterior is available in closed form.

The Beta-Bernoulli pair

Kronos motion — family decades

For a success probability p with a Beta(α, β) prior and k successes in n Bernoulli trials, the posterior is Beta(α + k, β + n − k). The prior parameters act like pseudo-counts of prior successes and failures, and the data simply add to them. This is the cleanest illustration of Bayesian learning.

Other standard pairs

Gamma prior with Poisson likelihood → Gamma posterior (event rates).
Normal prior with normal likelihood, known variance → Normal posterior.
Dirichlet prior with multinomial likelihood → Dirichlet posterior (categories).

python

# Beta-Bernoulli update
alpha, beta = 2, 2         # prior pseudo-counts
k, n = 7, 10               # data
post_alpha, post_beta = alpha+k, beta+(n-k)
post_mean = post_alpha/(post_alpha+post_beta)
print(round(post_mean,4))  # 0.6429

Why conjugacy is useful

Closed-form updates are fast, interpretable, and exact, which makes conjugate models ideal for streaming data and for building intuition. The pseudo-count interpretation shows precisely how prior strength trades off against data volume.

Limits

Conjugate priors exist only for special likelihoods and may not match genuine prior belief. When the natural prior is not conjugate, or the model is complex, one abandons closed form and samples the posterior numerically with MCMC.