On the Relationship Between Markov chain Monte Carlo Methods for Model Uncertainty
- 1 June 2001
- journal article
- Published by Informa UK Limited in Journal of Computational and Graphical Statistics
- Vol. 10 (2), 230-248
- https://doi.org/10.1198/10618600152627924
Abstract
This article considers Markov chain computational methods for incorporating uncer- tainty about the dimension of a parameter when performing inference within a Bayesian setting. A general class of methods is proposed for performing such computations, based upon a product space representation of the problem which is similar to that of Carlin and Chib. It is shown that all of the existing algorithms for incorporation of model uncertainty into Markov chain Monte Carlo (MCMC) can be derived as special cases of this gen- eral class of methods. In particular, we show that the popular reversible jump method is obtained when a special form of Metropolis-Hastings (M-H) algorithm is applied to the product space. Furthermore, the Gibbs sampling method and the variable selection method are shown to derive straightforwardly from the general framework. We believe that these new relationships between methods, which were until now seen as diverse procedures, are an important aid to the understanding of MCMC model selection procedures and may assist in the future development of improved procedures. Our discussion also sheds some light upon the important issues of "pseudo-prior" selection in the case of the Carlin and Chib sampler and choice of proposal distribution in the case of reversible jump. Finally, we pro- pose efficient reversible jump proposal schemes that take advantage of any analytic structure that may be present in the model. These proposal schemes are compared with a standard reversible jump scheme for the problem of model order uncertainty in autoregressive time series, demonstrating the improvements which can be achieved through careful choice of proposals.Keywords
This publication has 23 references indexed in Scilit:
- Automatic Bayesian Curve FittingJournal of the Royal Statistical Society Series B: Statistical Methodology, 1998
- Analysis of multivariate probit modelsBiometrika, 1998
- Bayesian estimation of an autoregressive model using Markov chain Monte CarloJournal of Econometrics, 1996
- Markov chain Monte Carlo in conditionally Gaussian state space modelsBiometrika, 1996
- Prediction Via Orthogonalized Model MixingJournal of the American Statistical Association, 1996
- Marginal Likelihood from the Gibbs OutputJournal of the American Statistical Association, 1995
- Inference from Iterative Simulation Using Multiple SequencesStatistical Science, 1992
- Sampling-Based Approaches to Calculating Marginal DensitiesJournal of the American Statistical Association, 1990
- A candidate's formula: A curious result in Bayesian predictionBiometrika, 1989
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of ImagesIeee Transactions On Pattern Analysis and Machine Intelligence, 1984