The Balanced Accuracy and Its Posterior Distribution
Top Cited Papers
- 1 August 2010
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3121-3124
- https://doi.org/10.1109/icpr.2010.764
Abstract
Evaluating the performance of a classification algorithm critically requires a measure of the degree to which unseen examples have been identified with their correct class labels. In practice, generalizability is frequently estimated by averaging the accuracies obtained on individual cross-validation folds. This procedure, however, is problematic in two ways. First, it does not allow for the derivation of meaningful confidence intervals. Second, it leads to an optimistic estimate when a biased classifier is tested on an imbalanced dataset. We show that both problems can be overcome by replacing the conventional point estimate of accuracy by an estimate of the posterior distribution of the balanced accuracy.Keywords
This publication has 5 references indexed in Scilit:
- The Binormal Assumption on Precision-Recall CurvesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2010
- A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reductionGenetic Epidemiology, 2007
- Applying Support Vector Machines to Imbalanced DatasetsLecture Notes in Computer Science, 2004
- The class imbalance problem: A systematic study1Intelligent Data Analysis, 2002
- SMOTE: Synthetic Minority Over-sampling TechniqueJournal of Artificial Intelligence Research, 2002