Unsupervised Domain Adaptation From Axial to Short-Axis Multi-Slice Cardiac MR Images by Incorporating Pretrained Task Networks
- 20 January 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 40 (10), 2939-2953
- https://doi.org/10.1109/tmi.2021.3052972
Abstract
Anisotropic multi-slice Cardiac Magnetic Resonance (CMR) Images are conventionally acquired in patient-specific short-axis (SAX) orientation. In specific cardiovascular diseases that affect right ventricular (RV) morphology, acquisitions in standard axial (AX) orientation are preferred by some investigators, due to potential superiority in RV volume measurement for treatment planning. Unfortunately, due to the rare occurrence of these diseases, data in this domain is scarce. Recent research in deep learning-based methods mainly focused on SAX CMR images and they had proven to be very successful. In this work, we show that there is a considerable domain shift between AX and SAX images, and therefore, direct application of existing models yield sub-optimal results on AX samples. We propose a novel unsupervised domain adaptation approach, which uses task-related probabilities in an attention mechanism. Beyond that, cycle consistency is imposed on the learned patient-individual 3D rigid transformation to improve stability when automatically re-sampling the AX images to SAX orientations. The network was trained on 122 registered 3D AX-SAX CMR volume pairs from a multi-centric patient cohort. A mean 3D Dice of 0.86 ± 0.06 for the left ventricle, 0.65 ± 0.08 for the myocardium, and 0.77 ± 0.10 for the right ventricle could be achieved. This is an improvement of 25% in Dice for RV in comparison to direct application on axial slices. To conclude, our pre-trained task module has neither seen CMR images nor labels from the target domain, but is able to segment them after the domain gap is reduced. Code: https://github.com/Cardio-AI/3d-mri-domain-adaptation.Other Versions
Funding Information
- Informatics for Life Project through the Klaus Tschira Foundation
- Competence Network for Congenital Heart Defects through the Federal Ministry of Education and Research (01GI0601)
- German Centre for Cardiovascular Research
This publication has 36 references indexed in Scilit:
- BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic SegmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image SegmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- U-Net: Convolutional Networks for Biomedical Image SegmentationPublished by Springer Science and Business Media LLC ,2015
- Right ventricle segmentation from cardiac MRI: A collation studyMedical Image Analysis, 2015
- A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR imagesMedical Image Analysis, 2013
- Impact of Gender and Age on Cardiovascular Function Late After Repair of Tetralogy of FallotCirculation: Cardiovascular Imaging, 2011
- Comparison of Accuracy of Axial Slices Versus Short-Axis Slices for Measuring Ventricular Volumes by Cardiac Magnetic Resonance in Patients With Corrected Tetralogy of FallotThe American Journal of Cardiology, 2009
- Axial slices compared to short-axis slices for measurement of right and left ventricle volumes of patients with corrected Tetralogy of Fallot and Ebstein's anomaly by cardiac magnetic resonanceJournal of Cardiovascular Magnetic Resonance, 2009
- Comparison of right ventricular volume measurements between axial and short axis orientation using steady‐state free precession magnetic resonance imagingJournal of Magnetic Resonance Imaging, 2003
- A method for registration of 3-D shapesIEEE Transactions on Pattern Analysis and Machine Intelligence, 1992