Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool
Top Cited Papers
Open Access
- 12 August 2015
- journal article
- research article
- Published by Springer Science and Business Media LLC in BMC Medical Imaging
- Vol. 15 (1), 1-28
- https://doi.org/10.1186/s12880-015-0068-x
Abstract
Medical Image segmentation is an important image processing step. Comparing images to evaluate the quality of segmentation is an essential part of measuring progress in this research area. Some of the challenges in evaluating medical segmentation are: metric selection, the use in the literature of multiple definitions for certain metrics, inefficiency of the metric calculation implementations leading to difficulties with large volumes, and lack of support for fuzzy segmentation by existing metrics. First we present an overview of 20 evaluation metrics selected based on a comprehensive literature review. For fuzzy segmentation, which shows the level of membership of each voxel to multiple classes, fuzzy definitions of all metrics are provided. We present a discussion about metric properties to provide a guide for selecting evaluation metrics. Finally, we propose an efficient evaluation tool implementing the 20 selected metrics. The tool is optimized to perform efficiently in terms of speed and required memory, also if the image size is extremely large as in the case of whole body MRI or CT volume segmentation. An implementation of this tool is available as an open source project. We propose an efficient evaluation tool for 3D medical image segmentation using 20 evaluation metrics and provide guidelines for selecting a subset of these metrics that is suitable for the data and the segmentation task.Keywords
This publication has 58 references indexed in Scilit:
- Computerized Segmentation and Characterization of Breast Lesions in Dynamic Contrast-Enhanced MR Images Using Fuzzy c-Means Clustering and Snake AlgorithmComputational and Mathematical Methods in Medicine, 2012
- Defuzzification of spatial fuzzy sets by feature distance minimizationImage and Vision Computing, 2011
- Review of shape representation and description techniquesPattern Recognition, 2004
- Mahalanobis distanceResonance, 1999
- The use of the area under the ROC curve in the evaluation of machine learning algorithmsPattern Recognition, 1997
- Comparing partitionsJournal of Classification, 1985
- Information retrievalACM SIGIR Forum, 1983
- Objective Criteria for the Evaluation of Clustering MethodsJournal of the American Statistical Association, 1971
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960
- Measures of the Amount of Ecologic Association Between SpeciesEcology, 1945