Estimating classification probabilities in high-dimensional diagnostic studies
Open Access
- 22 July 2011
- journal article
- research article
- Published by Oxford University Press (OUP) in Bioinformatics
- Vol. 27 (18), 2563-2570
- https://doi.org/10.1093/bioinformatics/btr434
Abstract
Motivation: Classification algorithms for high-dimensional biological data like gene expression profiles or metabolomic fingerprints are typically evaluated by the number of misclassifications across a test dataset. However, to judge the classification of a single case in the context of clinical diagnosis, we need to assess the uncertainties associated with that individual case rather than the average accuracy across many cases. Reliability of individual classifications can be expressed in terms of class probabilities. While classification algorithms are a well-developed area of research, the estimation of class probabilities is considerably less progressed in biology, with only a few classification algorithms that provide estimated class probabilities. Results: We compared several probability estimators in the context of classification of metabolomics profiles. Evaluation criteria included sparseness biases, calibration of the estimator, the variance of the estimator and its performance in identifying highly reliable classifications. We observed that several of them display artifacts that compromise their use in practice. Classification probabilities based on a combination of local cross-validation error rates and monotone regression prove superior in metabolomic profiling. Availability: The source code written in R is freely available at http://compdiag.uni-regensburg.de/software/probEstimation.shtml. Contact:inka.appel@klinik.uni-regensburg.deKeywords
This publication has 17 references indexed in Scilit:
- Detection of autosomal dominant polycystic kidney disease by NMR spectroscopic fingerprinting of urineKidney International, 2011
- The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive modelsNature Biotechnology, 2010
- Outcome prediction based on microarray analysis: a critical perspective on methodsBMC Bioinformatics, 2009
- Improved classification accuracy in 1- and 2-dimensional NMR metabolomics data using the variance stabilising generalised logarithm transformationBMC Bioinformatics, 2007
- Taking gene-expression profiling to the clinic: when will molecular signatures become relevant to patient care?Nature Reviews Cancer, 2007
- Interpretation of microarray data in cancerBritish Journal of Cancer, 2007
- A protocol for building and evaluating predictors of disease state based on microarray dataBioinformatics, 2005
- Predicting gene regulation by sigma factors in Bacillus subtilis from genome-wide dataBioinformatics, 2004
- Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression DataJournal of the American Statistical Association, 2002
- The Well-Calibrated BayesianJournal of the American Statistical Association, 1982