Deep learning-based fetoscopic mosaicking for field-of-view expansion
Open Access
- 17 August 2020
- journal article
- research article
- Published by Springer Science and Business Media LLC in International Journal of Computer Assisted Radiology and Surgery
- Vol. 15 (11), 1807-1816
- https://doi.org/10.1007/s11548-020-02242-8
Abstract
Purpose Fetoscopic laser photocoagulation is a minimally invasive surgical procedure used to treat twin-to-twin transfusion syndrome (TTTS), which involves localization and ablation of abnormal vascular connections on the placenta to regulate the blood flow in both fetuses. This procedure is particularly challenging due to the limited field of view, poor visibility, occasional bleeding, and poor image quality. Fetoscopic mosaicking can help in creating an image with the expanded field of view which could facilitate the clinicians during the TTTS procedure. Methods We propose a deep learning-based mosaicking framework for diverse fetoscopic videos captured from different settings such as simulation, phantoms, ex vivo, and in vivo environments. The proposed mosaicking framework extends an existing deep image homography model to handle video data by introducing the controlled data generation and consistent homography estimation modules. Training is performed on a small subset of fetoscopic images which are independent of the testing videos. Results We perform both quantitative and qualitative evaluations on 5 diverse fetoscopic videos (2400 frames) that captured different environments. To demonstrate the robustness of the proposed framework, a comparison is performed with the existing feature-based and deep image homography methods. Conclusion The proposed mosaicking framework outperformed existing methods and generated meaningful mosaic, while reducing the accumulated drift, even in the presence of visual challenges such as specular highlights, reflection, texture paucity, and low video resolution.Keywords
Funding Information
- Wellcome/EPSRC (203145Z/16/Z)
- Engineering and Physical Sciences Research Council (EP/P027938/1, EP/R004080/1)
- Engineering and Physical Sciences Research Council (NS/A000027/1)
- H2020 Future and Emerging Technologies (GA 863146)
- Royal Academy of Engineering Chair in Emerging Technologies (CiET1819/2/36)
- Engineering and Physical Sciences Research Council (EP/P012841/1)
- Medtronic/Royal Academy of Engineering Research Chair (RCSRF1819/7/34)
This publication has 21 references indexed in Scilit:
- Probabilistic visual and electromagnetic data fusion for robust drift-free sequential mosaicking: application to fetoscopyJournal of Medical Imaging, 2018
- Stable Image Registration for In-Vivo Fetoscopic Panorama ReconstructionJournal of Imaging, 2018
- A Continuum Robot and Control Interface for Surgical Assist in Fetoscopic InterventionsIEEE Robotics and Automation Letters, 2017
- Real-time mosaicing of fetoscopic videos using SIFTPublished by SPIE-Intl Soc Optical Eng ,2016
- The vascular anastomoses in monochorionic twin pregnancies and their clinical consequencesAmerican Journal of Obstetrics and Gynecology, 2012
- Twin-to-twin transfusion syndrome (TTTS)*jpme, 2010
- Automatic Panoramic Image Stitching using Invariant FeaturesInternational Journal of Computer Vision, 2006
- SURF: Speeded Up Robust FeaturesLecture Notes in Computer Science, 2006
- Consider the Lowly 2 × 2 MatrixPublished by Elsevier BV ,2003
- Alternative technique for Nd : YAG laser coagulation in twin‐to‐twin transfusion syndrome with anterior placentaUltrasound in Obstetrics & Gynecology, 1998