Elastic SCAD as a novel penalization method for SVM classification tasks in high-dimensional data
Open Access
- 9 May 2011
- journal article
- research article
- Published by Springer Science and Business Media LLC in BMC Bioinformatics
- Vol. 12 (1), 138
- https://doi.org/10.1186/1471-2105-12-138
Abstract
Classification and variable selection play an important role in knowledge discovery in high-dimensional data. Although Support Vector Machine (SVM) algorithms are among the most powerful classification and prediction methods with a wide range of scientific applications, the SVM does not include automatic feature selection and therefore a number of feature selection procedures have been developed. Regularisation approaches extend SVM to a feature selection method in a flexible way using penalty functions like LASSO, SCAD and Elastic Net. We propose a novel penalty function for SVM classification tasks, Elastic SCAD, a combination of SCAD and ridge penalties which overcomes the limitations of each penalty alone. Since SVM models are extremely sensitive to the choice of tuning parameters, we adopted an interval search algorithm, which in comparison to a fixed grid search finds rapidly and more precisely a global optimal solution. Feature selection methods with combined penalties (Elastic Net and Elastic SCAD SVMs) are more robust to a change of the model complexity than methods using single penalties. Our simulation study showed that Elastic SCAD SVM outperformed LASSO (L1) and SCAD SVMs. Moreover, Elastic SCAD SVM provided sparser classifiers in terms of median number of features selected than Elastic Net SVM and often better predicted than Elastic Net in terms of misclassification error. Finally, we applied the penalization methods described above on four publicly available breast cancer data sets. Elastic SCAD SVM was the only method providing robust classifiers in sparse and non-sparse situations. The proposed Elastic SCAD SVM algorithm provides the advantages of the SCAD penalty and at the same time avoids sparsity limitations for non-sparse data. We were first to demonstrate that the integration of the interval search algorithm and penalized SVM classification techniques provides fast solutions on the optimization of tuning parameters. The penalized SVM classification algorithms as well as fixed grid and interval search for finding appropriate tuning parameters were implemented in our freely available R package 'penalizedSVM'. We conclude that the Elastic SCAD SVM is a flexible and robust tool for classification and feature selection tasks for high-dimensional data such as microarray data sets.Keywords
This publication has 25 references indexed in Scilit:
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- Integration of pathway knowledge into a reweighted recursive feature elimination approach for risk stratification of cancer patientsBioinformatics, 2010
- Feature Selection and Classification of MAQC-II Breast Cancer and Multiple Myeloma Microarray Gene Expression DataPLOS ONE, 2009
- Measures to Summarize and Compare the Predictive Capacity of MarkersThe International Journal of Biostatistics, 2009
- Microarray technology: beyond transcript profiling and genotype analysisNature Reviews Genetics, 2006
- Regularization and Variable Selection Via the Elastic NetJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- A Gene-Expression Signature as a Predictor of Survival in Breast CancerThe New England Journal of Medicine, 2002
- Gene expression profiling predicts clinical outcome of breast cancerNature, 2002
- Computational analysis of microarray dataNature Reviews Genetics, 2001
- Wrappers for feature subset selectionArtificial Intelligence, 1997