A lasso for hierarchical interactions
Top Cited Papers
Open Access
- 1 June 2013
- journal article
- Published by Institute of Mathematical Statistics in The Annals of Statistics
- Vol. 41 (3), 1111-1141
- https://doi.org/10.1214/13-aos1096
Abstract
We add a set of convex constraints to the lasso to produce sparse interaction models that honor the hierarchy restriction that an interaction only be included in a model if one or both variables are marginally important. We give a precise characterization of the effect of this hierarchy constraint, prove that hierarchy holds with probability one and derive an unbiased estimate for the degrees of freedom of our estimator. A bound on this estimate reveals the amount of fitting “saved” by the hierarchy constraint. We distinguish between parameter sparsity—the number of nonzero coefficients—and practical sparsity—the number of raw variables one must measure to make a new prediction. Hierarchy focuses on the latter, which is more closely tied to important data collection concerns such as cost, time and effort. We develop an algorithm, available in the R package hierNet, and perform an empirical study of our method.Keywords
Other Versions
This publication has 29 references indexed in Scilit:
- Optimization with Sparsity-Inducing PenaltiesFoundations and Trends® in Machine Learning, 2011
- Screen and clean: a tool for identifying interactions in genome‐wide association studiesGenetic Epidemiology, 2010
- On the “degrees of freedom” of the lassoThe Annals of Statistics, 2007
- Genotypic predictors of human immunodeficiency virus type 1 drug resistanceProceedings of the National Academy of Sciences of the United States of America, 2006
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Better Subset Regression Using the Nonnegative GarroteTechnometrics, 1995
- Variable Selection via Gibbs SamplingJournal of the American Statistical Association, 1993
- Multivariate Adaptive Regression SplinesThe Annals of Statistics, 1991
- How Biased is the Apparent Error Rate of a Prediction Rule?Journal of the American Statistical Association, 1986
- Estimation of the Mean of a Multivariate Normal DistributionThe Annals of Statistics, 1981