Importance sampling and effective sample size

Let a normalized target density be $\pi$ and proposal density be $q$ on a common base measure. Require proposal positivity wherever the target contribution matters. For independent draws from $q$, multiplying $f(X)$ by $\pi(X)/q(X)$ changes its proposal expectation into the target expectation. Absolute integrability supports the expectation identity and a law of large numbers; a finite second moment of the weighted integrand is needed for the usual independent-sample variance and central limit analysis. Correct algebra does not ensure those moments are finite.

A proposal with lighter tails than the target can produce enormous weights in rare regions. Drawing more points in an already overrepresented region does not fix a missing-support defect, because its contribution is never seen. Proposal design should address the integrand too: a rare-event probability and a posterior mean can need different proposals. On finite support all relevant moments are finite if every target state has positive proposal probability, but the variance can still be so large that a small sample is practically uninformative.

For an unnormalized target $\widetilde\pi$ with finite positive integral, compute weights $w_i=\widetilde\pi(X_i)/q(X_i)$ and normalize them by their sum. The ratio of weighted function sum to weight sum cancels the unknown target normalizer. Under proposal support coverage and the relevant integrability, separate laws of large numbers give consistency. The ratio is generally biased for finite sample size, even when the numerator and denominator separately estimate their integrals without bias. Denominator zero must also be excluded or handled as a failed sample.

Evaluate weights on the logarithmic scale when products of many densities underflow. Subtracting a common maximum log weight before exponentiating preserves normalized weights, although it does not repair inadequate overlap. The sample average of unnormalized weights estimates the target normalizer if all its constants were retained. Self-normalization can obscure those constants deliberately for an expectation, so a routine returning only normalized weights does not automatically provide a marginal likelihood estimate. Keep the target and proposal density conventions explicit.

Weight ESS is largest when the observed weights are equal and approaches one when one draw dominates. It is a convenient warning about concentration, but it is not a universal count of independent target observations for every function. Error also depends on how the chosen function varies with the weights, and an unseen important region can leave all observed weights deceptively similar. Distinguish this diagnostic from autocorrelation-based MCMC ESS, which measures a different source of Monte Carlo inefficiency.

A practical investigation repeats sampling with different seeds and stronger proposals, examines the largest weights, and checks estimates against analytical cases or another method. Tail diagnostics can warn about unstable importance ratios, but finite diagnostics cannot certify complete support coverage or finite population moments. If the estimate changes substantially after one exceptional draw, treat the earlier apparent precision cautiously. Increasing proposal coverage or changing the estimator may be more effective than simply reporting a larger nominal sample size with the same unstable weighting mechanism.