Marginal Likelihood Estimation with the Cross-Entropy Method
- 1 May 2012
- preprint
- Published by Elsevier BV in SSRN Electronic Journal
Abstract
We consider an adaptive importance sampling approach to estimating the marginal likelihood, a quantity that is fundamental in Bayesian model comparison and Bayesian model averaging. This approach is motivated by the difficulty of obtaining an accurate estimate through existing algorithms that use Markov chain Monte Carlo (MCMC) draws, where the draws are typically costly to obtain and highly correlated in high-dimensional settings. In contrast, we use the cross-entropy (CE) method, a versatile adaptive Monte Carlo algorithm originally developed for rare-event simulation. The main advantage of the importance sampling approach is that random samples can be obtained from some convenient density with little additional costs. As we are generating independent draws instead of correlated MCMC draws, the increase in simulation effort is much smaller should one wish to reduce the numerical standard error of the estimator. Moreover, the importance density derived via the CE method is grounded in information theory, and therefore, is in a well-defined sense optimal. We demonstrate the utility of the proposed approach by two empirical applications involving women’s labor market participation and U.S. macroeconomic time series. In both applications the proposed CE method compares favorably to existing estimators.This publication has 37 references indexed in Scilit:
- Bayesian Multivariate Time Series Methods for Empirical MacroeconomicsFoundations and Trends® in Econometrics, 2009
- Efficient simulation and integrated likelihood estimation in state space modelsInternational Journal of Mathematical Modelling and Numerical Optimisation, 2009
- Information and Entropy Econometrics — A Review and SynthesisFoundations and Trends® in Econometrics, 2007
- Measuring the Effects of Monetary Policy: A Factor-Augmented Vector Autoregressive (FAVAR) ApproachThe Quarterly Journal of Economics, 2005
- The Cross-Entropy Method for Network Reliability EstimationAnnals of Operations Research, 2005
- Marginal Likelihood From the Metropolis–Hastings OutputJournal of the American Statistical Association, 2001
- Using simulation methods for bayesian econometric models: inference, development,and communicationEconometric Reviews, 1999
- Marginal Likelihood from the Gibbs OutputJournal of the American Statistical Association, 1995
- A compendium to information theory in economics and econometricsEconometric Reviews, 1993
- Economics of Information SystemsJournal of the American Statistical Association, 1971