Moments and Generating Functions
Moments summarize a distribution's shape, and generating functions package all of them into a single transform.
What a moment is
The k-th raw moment is E[Xᵏ]; the k-th central moment is E[(X − μ)ᵏ]. The first raw moment is the mean; the second central moment is the variance. Higher central moments describe shape.
- Skewness (standardized third moment): asymmetry of the distribution.
- Kurtosis (standardized fourth moment): weight of the tails relative to a normal.
Moment generating function
The moment generating function (MGF) is M(t) = E[e^{tX}], when it exists near t = 0. Its name comes from the fact that its derivatives at zero produce the moments: M^{(k)}(0) = E[Xᵏ]. Expanding e^{tX} as a power series shows why.
Why MGFs are useful
The MGF of a sum of independent variables is the product of their MGFs, which makes it a clean tool for finding the distribution of sums. Because a distribution is uniquely determined by its MGF (when it exists), matching MGFs is a rigorous way to identify a limiting distribution — one route to proving the central limit theorem.
When the MGF fails
Heavy-tailed distributions, such as the Cauchy, have no MGF because E[e^{tX}] diverges. The characteristic function E[e^{itX}], using imaginary exponent, always exists and serves the same role, which is why advanced treatments prefer it.
In practice, moments are estimated from sample averages, and the method of moments equates sample and theoretical moments to fit distribution parameters.