Shading-based refinement on volumetric signed distance functions
- 27 July 2015
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 34 (4), 1-14
- https://doi.org/10.1145/2766887
Abstract
We present a novel method to obtain fine-scale detail in 3D reconstructions generated with low-budget RGB-D cameras or other commodity scanning devices. As the depth data of these sensors is noisy, truncated signed distance fields are typically used to regularize out the noise, which unfortunately leads to over-smoothed results. In our approach, we leverage RGB data to refine these reconstructions through shading cues, as color input is typically of much higher resolution than the depth data. As a result, we obtain reconstructions with high geometric detail, far beyond the depth resolution of the camera itself. Our core contribution is shading-based refinement directly on the implicit surface representation, which is generated from globally-aligned RGB-D images. We formulate the inverse shading problem on the volumetric distance field, and present a novel objective function which jointly optimizes for fine-scale surface geometry and spatially-varying surface reflectance. In order to enable the efficient reconstruction of sub-millimeter detail, we store and process our surface using a sparse voxel hashing scheme which we augment by introducing a grid hierarchy. A tailored GPU-based Gauss-Newton solver enables us to refine large shape models to previously unseen resolution within only a few seconds.Keywords
This publication has 47 references indexed in Scilit:
- Efficient GPU Data Structures and Methods to Solve Sparse Linear Systems in Dynamics ApplicationsComputer Graphics Forum, 2012
- Coherent Spatiotemporal Filtering, Upsampling and Rendering of RGBZ VideosComputer Graphics Forum, 2012
- RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environmentsThe International Journal of Robotics Research, 2012
- Multiview face capture using polarized spherical gradient illuminationACM Transactions on Graphics, 2011
- Building Rome in a dayCommunications of the ACM, 2011
- Joint bilateral upsamplingACM Transactions on Graphics, 2007
- Photo tourismACM Transactions on Graphics, 2006
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004
- Real-time 3D model acquisitionACM Transactions on Graphics, 2002
- Shape-from-shading: a surveyIEEE Transactions on Pattern Analysis and Machine Intelligence, 1999