An Importance Sampling Estimate

Estimate a rare-event probability far more efficiently by sampling from a shifted distribution and reweighting.

Problem

Importance sampling estimates expectations under one distribution by sampling from another and correcting with weights. It shines for rare events, where naive Monte Carlo almost never lands a hit. We estimate the probability that a standard normal exceeds 4, a tail event of about 3e-5.

Method

Sample from a proposal q centered near the rare region (a normal shifted to mean 4), then weight each sample by p(x)/q(x), the ratio of target to proposal densities. The weighted average of the indicator gives the estimate with far lower variance than direct sampling.

import numpy as np
rng=np.random.default_rng(5)
from scipy.stats import norm
N=100000; mu=4.0
x=rng.normal(mu,1,N)                 # proposal q ~ N(4,1)
w=norm.pdf(x,0,1)/norm.pdf(x,mu,1)   # importance weights
est=np.mean((x>4)*w)
true=1-norm.cdf(4)
print('IS estimate',est,'true',round(true,3e-1*0+8))

Result

The importance-sampling estimate lands close to the true tail probability of about 3.17e-5 using a modest sample count, whereas direct sampling from N(0,1) would need on the order of a million draws to see even a handful of exceedances. The weights correct for the fact that we oversampled the tail. A poorly chosen proposal, one that misses the important region, produces high-variance or biased estimates, so the proposal design is the whole art.

• Good proposals concentrate samples where the integrand is large, slashing variance for rare events.
• Weights with heavy tails signal a bad proposal and can wreck the estimate despite a low apparent error.
• Kronos applies importance sampling to rare-failure and neutron-tail estimates where direct Monte Carlo is impractical.