3D Graph Neural Networks for RGBD Semantic Segmentation
- 1 October 2017
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5209-5218
- https://doi.org/10.1109/iccv.2017.556
Abstract
RGBD semantic segmentation requires joint reasoning about 2D appearance and 3D geometric information. In this paper we propose a 3D graph neural network (3DGNN) that builds a k-nearest neighbor graph on top of 3D point cloud. Each node in the graph corresponds to a set of points and is associated with a hidden representation vector initialized with an appearance feature extracted by a unary CNN from 2D images. Relying on recurrent functions, every node dynamically updates its hidden representation based on the current status and incoming messages from its neighbors. This propagation model is unrolled for a certain number of time steps and the final per-node representation is used for predicting the semantic class of each pixel. We use back-propagation through time to train the model. Extensive experiments on NYUD2 and SUN-RGBD datasets demonstrate the effectiveness of our approach.Keywords
This publication has 23 references indexed in Scilit:
- Deep Residual Learning for Image RecognitionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D ImagesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet ClassificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Geodesic Convolutional Neural Networks on Riemannian ManifoldsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Unsupervised Joint Feature Learning and Encoding for RGB-D Scene LabelingIEEE Transactions on Image Processing, 2015
- SUN RGB-D: A RGB-D scene understanding benchmark suitePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Improved Semantic Representations From Tree-Structured Long Short-Term Memory NetworksPublished by Association for Computational Linguistics (ACL) ,2015
- Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic SegmentationInternational Journal of Computer Vision, 2014
- The Graph Neural Network ModelIEEE Transactions on Neural Networks, 2008
- Long Short-Term MemoryNeural Computation, 1997