Multi-view Convolutional Neural Networks for 3D Shape Recognition
Top Cited Papers
- 1 December 2015
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 945-953
- https://doi.org/10.1109/iccv.2015.114
Abstract
A longstanding question in computer vision concerns the representation of 3D shapes for recognition: should 3D shapes be represented with descriptors operating on their native 3D formats, such as voxel grid or polygon mesh, or can they be effectively represented with view-based descriptors? We address this question in the context of learning to recognize 3D shapes from a collection of their rendered views on 2D images. We first present a standard CNN architecture trained to recognize the shapes' rendered views independently of each other, and show that a 3D shape can be recognized even from a single view at an accuracy far higher than using state-of-the-art 3D shape descriptors. Recognition rates further increase when multiple views of the shapes are provided. In addition, we present a novel CNN architecture that combines information from multiple views of a 3D shape into a single and compact shape descriptor offering even better recognition performance. The same architecture can be applied to accurately recognize human hand-drawn sketches of shapes. We conclude that a collection of 2D views can be highly informative for 3D shape recognition and is amenable to emerging CNN architectures and their derivatives.Keywords
Other Versions
This publication has 25 references indexed in Scilit:
- Sketch classification and classification-driven analysis using Fisher vectorsACM Transactions on Graphics, 2014
- Rich Feature Hierarchies for Accurate Object Detection and Semantic SegmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Fisher Vector Faces in the WildPublished by British Machine Vision Association and Society for Pattern Recognition ,2013
- How do humans sketch objects?ACM Transactions on Graphics, 2012
- Sketch-based shape retrievalACM Transactions on Graphics, 2012
- Discriminative Sketch-based 3D Model Retrieval via Robust Shape MatchingComputer Graphics Forum, 2011
- Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contoursPublished by Association for Computing Machinery (ACM) ,2010
- Hough Transform and 3D SURF for Robust Three Dimensional ClassificationLecture Notes in Computer Science, 2010
- Extended Gaussian imagesProceedings of the IEEE, 1984
- The singularities of the visual mappingBiological Cybernetics, 1976