Combining Estimates in Regression and Classification
- 1 December 1996
- journal article
- research article
- Published by Informa UK Limited in Journal of the American Statistical Association
- Vol. 91 (436), 1641-1650
- https://doi.org/10.1080/01621459.1996.10476733
Abstract
We consider the problem of how to combine a collection of general regression fit vectors to obtain a better predictive model. The individual fits may be from subset linear regression, ridge regression, or something more complex like a neural network. We develop a general framework for this problem and examine a cross-validation—based proposal called “model mix” or “stacking” in this context. We also derive combination methods based on the bootstrap and analytic methods and compare them in examples. Finally, we apply these ideas to classification problems where the estimated combination weights can yield insight into the structure of the problem.Keywords
This publication has 7 references indexed in Scilit:
- Varying-Coefficient ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 1993
- An Introduction to the BootstrapPublished by Springer Science and Business Media LLC ,1993
- Stacked generalizationNeural Networks, 1992
- A Simple Method for the Adjustment of Profile LikelihoodsJournal of the Royal Statistical Society Series B: Statistical Methodology, 1990
- Estimating the Error Rate of a Prediction Rule: Improvement on Cross-ValidationJournal of the American Statistical Association, 1983
- Hedonic housing prices and the demand for clean airJournal of Environmental Economics and Management, 1978
- Cross‐Validatory Choice and Assessment of Statistical PredictionsJournal of the Royal Statistical Society: Series B (Methodological), 1974