A bias correction for the minimum error rate in cross-validation
Open Access
- 1 June 2009
- journal article
- Published by Institute of Mathematical Statistics in The Annals of Applied Statistics
- Vol. 3 (2), 822-829
- https://doi.org/10.1214/08-aoas224
Abstract
Tuning parameters in supervised learning problems are often estimated by cross-validation. The minimum value of the cross-validation error can be biased downward as an estimate of the test error at that same value of the tuning parameter. We propose a simple method for the estimation of this bias that uses information from the cross-validation process. As a result, it requires essentially no additional computation. We apply our bias estimate to a number of popular classifiers in various settings, and examine its performance.Keywords
This publication has 6 references indexed in Scilit:
- Empirical Bayes Estimates for Large-Scale Prediction ProblemsJournal of the American Statistical Association, 2009
- Bias in error estimation when using cross-validation for model selectionBMC Bioinformatics, 2006
- Diagnosis of multiple cancer types by shrunken centroids of gene expressionProceedings of the National Academy of Sciences of the United States of America, 2002
- An Introduction to the BootstrapPublished by Springer Science and Business Media LLC ,1993
- Bootstrap Methods: Another Look at the JackknifeThe Annals of Statistics, 1979
- Asymptotics for and against cross-validationBiometrika, 1977