Expectation

The expectation of a random variable is its probability-weighted average, the long-run mean of repeated draws.

Definition

The expectation or mean of X is E[X] = Σ x p(x) for discrete variables and ∫ x f(x) dx for continuous ones. It is the center of mass of the distribution: the balance point of the probability weights.

Linearity

Kronos motion — long view

The single most useful property is linearity: E[aX + bY + c] = a E[X] + b E[Y] + c, for any random variables X and Y and constants a, b, c. Crucially, this holds even when X and Y are dependent. No independence assumption is needed.

Law of the unconscious statistician

The expectation of a function is E[g(X)] = Σ g(x) p(x) or ∫ g(x) f(x) dx. You do not need the distribution of g(X); you integrate g against the distribution of X directly. This is what makes Monte Carlo estimation possible.

Expectation of a product

For independent variables, E[XY] = E[X] E[Y]. When they are dependent the difference E[XY] − E[X]E[Y] is exactly the covariance, which is why covariance measures linear dependence.

Estimating expectations

The sample mean of n draws is an unbiased estimator of E[X], and by the law of large numbers it converges to E[X] as n grows. Every Monte Carlo estimate is at heart an approximate expectation: run the model many times, average the quantity of interest.

python

xs = [3,1,4,1,5,9,2,6]
mean = sum(xs)/len(xs)
print(mean)  # 3.875