Multi-modal RGB-D Image Segmentation from Appearance and Geometric Depth Maps
Open Access
- 15 May 2020
- journal article
- Published by Instituto Tecnologico Metropolitano (ITM) in TecnoLógicas
- Vol. 23 (48), 143-161
- https://doi.org/10.22430/22565337.1538
Abstract
Classical image segmentation algorithms exploit the detection of similarities and discontinuities of different visual cues to define and differentiate multiple regions of interest in images. However, due to the high variability and uncertainty of image data, producing accurate results is difficult. In other words, segmentation based just on color is often insufficient for a large percentage of real-life scenes. This work presents a novel multi-modal segmentation strategy that integrates depth and appearance cues from RGB-D images by building a hierarchical region-based representation, i.e., a multi-modal segmentation tree (MM-tree). For this purpose, RGB-D image pairs are represented in a complementary fashion by different segmentation maps. Based on color images, a color segmentation tree (C-tree) is created to obtain segmented and over-segmented maps. From depth images, two independent segmentation maps are derived by computing planar and 3D edge primitives. Then, an iterative region merging process can be used to locally group the previously obtained maps into the MM-tree. Finally, the top emerging MM-tree level coherently integrates the available information from depth and appearance maps. The experiments were conducted using the NYU-Depth V2 RGB-D dataset, which demonstrated the competitive results of our strategy compared to state-of-the-art segmentation methods. Specifically, using test images, our method reached average scores of 0.56 in Segmentation Covering and 2.13 in Variation of Information.Keywords
This publication has 27 references indexed in Scilit:
- A Global/Local Affinity Graph for Image SegmentationIEEE Transactions on Image Processing, 2015
- Graph-Based Segmentation for RGB-D Data Using 3-D Geometry Enhanced SuperpixelsIEEE Transactions on Cybernetics, 2014
- Enhanced Computer Vision With Microsoft Kinect Sensor: A ReviewIEEE Transactions on Cybernetics, 2013
- Perceptual Organization and Recognition of Indoor Scenes from RGB-D ImagesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Learning of perceptual grouping for object segmentation on RGB-D dataJournal of Visual Communication and Image Representation, 2013
- Segmentation using superpixels: A bipartite graph partitioning approachPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- RGB-(D) scene labeling: Features and algorithmsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- Indoor Segmentation and Support Inference from RGBD ImagesLecture Notes in Computer Science, 2012
- Contour Detection and Hierarchical Image SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2010
- Statistical region mergingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2004