Exact sampling for intractable probability distributions via a Bernoulli factory
Open Access
- 1 January 2012
- journal article
- Published by Institute of Mathematical Statistics in Electronic Journal of Statistics
- Vol. 6 (none), 10-37
- https://doi.org/10.1214/11-ejs663
Abstract
Many applications in the field of statistics require Markov chain Monte Carlo methods. Determining appropriate starting values and run lengths can be both analytically and empirically challenging. A desire to overcome these problems has led to the development of exact, or perfect, sampling algorithms which convert a Markov chain into an algorithm that produces i.i.d. samples from the stationary distribution. Unfortunately, very few of these algorithms have been developed for the distributions that arise in statistical applications, which typically have uncountable support. Here we study an exact sampling algorithm using a geometrically ergodic Markov chain on a general state space. Our work provides a significant reduction to the number of input draws necessary for the Bernoulli factory, which enables exact sampling via a rejection sampling approach. We illustrate the algorithm on a univariate Metropolis-Hastings sampler and a bivariate Gibbs sampler, which provide a proof of concept and insight into hyper-parameter selection. Finally, we illustrate the algorithm on a Bayesian version of the one-way random effects model with data from a styrene exposure study.Keywords
Other Versions
This publication has 32 references indexed in Scilit:
- Batch means and spectral variance estimators in Markov chain Monte CarloThe Annals of Statistics, 2010
- Markov Chain Monte Carlo: Can We Trust the Third Significant Figure?Statistical Science, 2008
- Using a Markov Chain to Construct a Tractable Approximation of an Intractable Probability DistributionScandinavian Journal of Statistics, 2006
- Fast simulation of new coins from oldThe Annals of Applied Probability, 2005
- A mixture representation of π with applications in Markov chain Monte Carlo and perfect samplingThe Annals of Applied Probability, 2004
- Perfect sampling using bounding chainsThe Annals of Applied Probability, 2004
- Sufficient burn-in for Gibbs samplers for a hierarchical random effects modelThe Annals of Statistics, 2004
- On the applicability of regenerative simulation in Markov chain Monte CarloBiometrika, 2002
- Minorization Conditions and Convergence Rates for Markov Chain Monte CarloJournal of the American Statistical Association, 1995
- Regeneration in Markov Chain SamplersJournal of the American Statistical Association, 1995