Spike and slab variable selection: Frequentist and Bayesian strategies
Open Access
- 1 April 2005
- journal article
- Published by Institute of Mathematical Statistics in The Annals of Statistics
- Vol. 33 (2), 730-773
- https://doi.org/10.1214/009053604000001147
Abstract
Variable selection in the linear regression model takes many apparent faces from both frequentist and Bayesian standpoints. In this paper we introduce a variable selection method referred to as a rescaled spike and slab model. We study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation. Specifically, we study the usefulness of continuous bimodal priors to model hypervariance parameters, and the effect scaling has on the posterior mean through its relationship to penalization. Several model selection strategies, some frequentist and some Bayesian in nature, are developed and studied theoretically. We demonstrate the importance of selective shrinkage for effective variable selection in terms of risk misclassification, and show this is achieved using the posterior from a rescaled spike and slab model. We also show how to verify a procedure’s ability to reduce model uncertainty in finite samples using a specialized forward selection strategy. Using this tool, we illustrate the effectiveness of rescaled spike and slab models in reducing model uncertainty.This publication has 26 references indexed in Scilit:
- Prediction via Orthogonalized Model MixingJournal of the American Statistical Association, 1996
- Bootstrap Model SelectionJournal of the American Statistical Association, 1996
- Consistent Variable Selection in Linear ModelsJournal of the American Statistical Association, 1995
- Variable Selection via Gibbs SamplingJournal of the American Statistical Association, 1993
- Linear Model Selection by Cross-validationJournal of the American Statistical Association, 1993
- The Little Bootstrap and other Methods for Dimensionality Selection in Regression: X-Fixed Prediction ErrorJournal of the American Statistical Association, 1992
- On the Distributional Properties of Model Selection CriteriaJournal of the American Statistical Association, 1992
- Variable Selection in Nonparametric Regression with Categorical CovariatesJournal of the American Statistical Association, 1992
- Bayesian Variable Selection in Linear RegressionJournal of the American Statistical Association, 1988
- Ridge Regression: Biased Estimation for Nonorthogonal ProblemsTechnometrics, 1970