We describe briefly the powerful simulation technique known as “importance sampling.” Importance sampling is a technique that allows you to use numerical simulation to explore events that, at first look, appear too rare to be reliably approximated numerically. The correctness of importance sampling follows almost immediately from the definition of a change of density. Like most mathematical techniques, importance sampling brings in its own concerns and controls that were not obvious in the original problem. To deal with these concerns (like picking the re-weighting to use) we will largely appeal to the ideas from “A Tutorial on the Cross-Entropy Method” Pieter-Tjerk de Boer, Dirk P Kroese, Shie Mannor, and Reuven Y Rubinstein, Annals of Operations Research, 2005 vol. 134 (1) pp. 19-67.To make things concrete we describe the application of the method to a very simplified version of the problem of modeling mortgage defaults. Our writeup re-derives most everything for clarity and can be found here: https://github.com/WinVector/Examples/tree/main/dfiles/ImportanceSampling.pdf
Data Scientist and trainer at Win Vector LLC. One of the authors of Practical Data Science with R.