The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)
Top Cited Papers
Open Access
- 4 December 2014
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 34 (10), 1993-2024
- https://doi.org/10.1109/tmi.2014.2377694
Abstract
In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients - manually annotated by up to four raters - and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.Keywords
This publication has 108 references indexed in Scilit:
- An Open Source Multivariate Framework for n-Tissue Segmentation with Evaluation on Public DataNeuroinformatics, 2011
- Multi-parametric neuroimaging reproducibility: A 3-T resource studyNeuroImage, 2011
- A reproducible evaluation of ANTs similarity metric performance in brain image registrationNeuroImage, 2011
- Segmentation of image ensembles via latent atlasesMedical Image Analysis, 2010
- Simulation of brain tumors in MR images for evaluation of segmentation efficacyMedical Image Analysis, 2009
- Online resource for validation of brain segmentation methodsNeuroImage, 2009
- New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1)European Journal of Cancer, 2009
- Multiparametric Tissue Characterization of Brain Neoplasms and Their Recurrence Using Pattern Classification of MR ImagesAcademic Radiology, 2008
- Lesion identification using unified segmentation-normalisation models and fuzzy clusteringNeuroImage, 2008
- On evaluating brain tissue classifiers without a ground truthNeuroImage, 2007