Shape Completion Using 3D-Encoder-Predictor CNNs and Shape Synthesis
Top Cited Papers
- 1 July 2017
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636919,p. 6545-6554
- https://doi.org/10.1109/cvpr.2017.693
Abstract
We introduce a data-driven approach to complete partial 3D shapes through a combination of volumetric deep neural networks and 3D shape synthesis. From a partially-scanned input shape, our method first infers a low-resolution - but complete - output. To this end, we introduce a 3D-Encoder-Predictor Network (3D-EPN) which is composed of 3D convolutional layers. The network is trained to predict and fill in missing data, and operates on an implicit surface representation that encodes both known and unknown space. This allows us to predict global structure in unknown areas at high accuracy. We then correlate these intermediary results with 3D geometry from a shape database at test time. In a final pass, we propose a patch-based 3D shape synthesis method that imposes the 3D geometry from these retrieved shapes as constraints on the coarsely-completed mesh. This synthesis process enables us to reconstruct fine-scale detail and generate high-resolution output while respecting the global mesh structure obtained by the 3D-EPN. Although our 3D-EPN outperforms state-of-the-art completion method, the main contribution in our work lies in the combination of a data-driven shape predictor and analytic 3D shape synthesis. In our results, we show extensive evaluations on a newly-introduced shape completion benchmark for both real-world and synthetic data.Keywords
This publication has 33 references indexed in Scilit:
- Database‐Assisted Object Retrieval for Real‐Time 3D ReconstructionComputer Graphics Forum, 2015
- Real-time shading-based refinement for consumer depth camerasACM Transactions on Graphics, 2014
- Approximate Symmetry Detection in Partial 3D MeshesComputer Graphics Forum, 2014
- Real-time 3D reconstruction at scale using voxel hashingACM Transactions on Graphics, 2013
- Scalable real-time volumetric surface reconstructionACM Transactions on Graphics, 2013
- Screened poisson surface reconstructionACM Transactions on Graphics, 2013
- Acquiring 3D indoor environments with variability and repetitionACM Transactions on Graphics, 2012
- A search-classify approach for cluttered indoor scene understandingACM Transactions on Graphics, 2012
- An interactive approach to semantic modeling of indoor scenes with an RGBD cameraACM Transactions on Graphics, 2012
- A robust hole-filling algorithm for triangular meshThe Visual Computer, 2007