SA-LuT-Nets: Learning Sample-Adaptive Intensity Lookup Tables for Brain Tumor Segmentation
- 3 February 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 40 (5), 1417-1427
- https://doi.org/10.1109/tmi.2021.3056678
Abstract
In clinics, the information about the appearance and location of brain tumors is essential to assist doctors in diagnosis and treatment. Automatic brain tumor segmentation on the images acquired by magnetic resonance imaging (MRI) is a common way to attain this information. However, MR images are not quantitative and can exhibit significant variation in signal depending on a range of factors, which increases the difficulty of training an automatic segmentation network and applying it to new MR images. To deal with this issue, this paper proposes to learn a sample-adaptive intensity lookup table (LuT) that dynamically transforms the intensity contrast of each input MR image to adapt to the following segmentation task. Specifically, the proposed deep SA-LuT-Net framework consists of a LuT module and a segmentation module, trained in an end-to-end manner: the LuT module learns a sample-specific nonlinear intensity mapping function through communication with the segmentation module, aiming at improving the final segmentation performance. In order to make the LuT learning sample-adaptive, we parameterize the intensity mapping function by exploring two families of non-linear functions (i.e., piece-wise linear and power functions) and predict the function parameters for each given sample. These sample-specific parameters make the intensity mapping adaptive to samples. We develop our SA-LuT-Nets separately based on two backbone networks for segmentation, i.e., DMFNet and the modified 3D Unet, and validate them on BRATS2018 and BRATS2019 datasets for brain tumor segmentation. Our experimental results clearly demonstrate the superior performance of the proposed SA-LuT-Nets using either single or multiple MR modalities. It not only significantly improves the two baselines (DMFNet and the modified 3D Unet), but also wins a set of state-of-the-art segmentation methods. Moreover, we show that, the LuTs learnt using one segmentation model could also be applied to improving the performance of another segmentation model, indicating the general segmentation information captured by LuTs.Funding Information
- Australian Research Council (ARC DP200103223)
- National Science Foundation of China (61876087, 62076135)
This publication has 36 references indexed in Scilit:
- Brain tumor segmentation with Deep Neural NetworksMedical Image Analysis, 2017
- Brain Tumor Segmentation Using Convolutional Neural Networks in MRI ImagesIEEE Transactions on Medical Imaging, 2016
- U-Net: Convolutional Networks for Biomedical Image SegmentationPublished by Springer Science and Business Media LLC ,2015
- DALSA: Domain Adaptation for Supervised Learning From Sparsely Annotated MR ImagesIEEE Transactions on Medical Imaging, 2015
- The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)IEEE Transactions on Medical Imaging, 2014
- Evaluating intensity normalization on MRIs of human brain with multiple sclerosisMedical Image Analysis, 2011
- Image enhancement for liver CT imagesPublished by SPIE-Intl Soc Optical Eng ,2009
- Influence of MRI acquisition protocols and image intensity normalization methods on texture classificationMagnetic Resonance Imaging, 2004
- Blind inverse gamma correctionIEEE Transactions on Image Processing, 2001
- New variants of a method of MRI scale standardizationIEEE Transactions on Medical Imaging, 2000