Uncertainty-Aware Annotation Protocol to Evaluate Deformable Registration Algorithms
Open Access
- 5 April 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 40 (8), 2053-2065
- https://doi.org/10.1109/tmi.2021.3070842
Abstract
Landmark correspondences are a widely used type of gold standard in image registration. However, the manual placement of corresponding points is subject to high inter-user variability in the chosen annotated locations and in the interpretation of visual ambiguities. In this paper, we introduce a principled strategy for the construction of a gold standard in deformable registration. Our framework: (i) iteratively suggests the most informative location to annotate next, taking into account its redundancy with previous annotations; (ii) extends traditional pointwise annotations by accounting for the spatial uncertainty of each annotation, which can either be directly specified by the user, or aggregated from pointwise annotations from multiple experts; and (iii) naturally provides a new strategy for the evaluation of deformable registration algorithms. Our approach is validated on four different registration tasks. The experimental results show the efficacy of suggesting annotations according to their informativeness, and an improved capacity to assess the quality of the outputs of registration algorithms. In addition, our approach yields, from sparse annotations only, a dense visualization of the errors made by a registration method. The source code of our approach supporting both 2D and 3D data is publicly available at https://github.com/LoicPeter/evaluation-deformable-registration.Funding Information
- European Research Council (677697)
- Engineering and Physical Sciences Research Council (EP/M020533/1)
- Chan Zuckerberg Initative DAF, an advised fund of Silicon Valley Community Foundation (2019-198101)
This publication has 23 references indexed in Scilit:
- An Interactive Approach to Solving Correspondence ProblemsInternational Journal of Computer Vision, 2013
- A novel in vivo atlas of human hippocampal subfields using high-resolution 3T magnetic resonance imagingNeuroImage, 2013
- A reference dataset for deformable image registration spatial accuracy evaluation using the COPDgene study archivePhysics in Medicine & Biology, 2013
- Fast parallel image registration on CPU and GPU for diagnostic classification of Alzheimer's diseaseFrontiers in Neuroscience, 2013
- FreeSurferNeuroImage, 2012
- Semi-automatic construction of reference standards for evaluation of image registrationMedical Image Analysis, 2011
- elastix: A Toolbox for Intensity-Based Medical Image RegistrationIEEE Transactions on Medical Imaging, 2009
- Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registrationNeuroImage, 2009
- A framework for evaluation of deformable image registration spatial accuracy using large landmark point setsPhysics in Medicine & Biology, 2009
- System for combined three‐dimensional morphological and molecular analysis of thick tissue specimensMicroscopy Research and Technique, 2002