Endomicroscopic video retrieval using mosaicing and visualwords

Abstract

In vivo pathology from endomicroscopy videos can be a challenge for many physicians. To ease this task, we propose a content-based video retrieval method providing, given a query video, relevant similar videos from an expert-annotated database. Our main contribution consists in revisiting the Bag of Visual Words method by weighting the contributions of the dense local regions according to the registration results of mosaicing. We perform a leave-one-patient-out k-nearest neighbors classification and show a significantly better accuracy (e.g. around 94% for 9 neighbors) when compared to using the video images independently. Less neighbors are needed to classify the queries and our signature summation technique reduces retrieval runtime.