Centimeter-wave Free-space Neural Time-of-Flight Imaging
Open Access
- 28 February 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 42 (1), 1-18
- https://doi.org/10.1145/3522671
Abstract
Depth sensors have emerged as a cornerstone sensor modality with diverse applications in personal hand-held devices, robotics, scientific imaging, autonomous vehicles, and more. In particular, correlation Time-of-Flight (ToF) sensors have found widespread adoption for meter-scale indoor applications such as object tracking and pose estimation. While they offer high depth resolution at competitive costs, the precision of these indirect ToF sensors is fundamentally limited by their modulation contrast, which is in turn limited by the effects of photo-conversion noise. In contrast, optical interferometric methods can leverage short illumination modulation wavelengths to achieve depth precision three orders of magnitude greater than ToF, but typically find their range restricted to the sub-centimeter. In this work, we merge concepts from both correlation ToF design and interferometric imaging; a step towards bridging the gap between these methods. We propose a computational ToF imaging method which optically computes the GHz ToF correlation signal in free space before photo-conversion. To acquire a depth map, we scan a scene point-wise and computationally unwrap the collected correlation measurements. Specifically, we repurpose electro-optical modulators used in optical communication for ToF imaging with centimeter-wave signals, and achieve all-optical correlation at 7.15 and 14.32 GHz modulation frequencies. While GHz modulation frequencies increase depth precision, these high modulation rates also pose a technical challenge. They result in dozens of wraps per meter which cannot be estimated robustly by existing phase unwrapping methods. We tackle this problem with a proposed segmentation-inspired phase unwrapping network, which exploits the correlation of adjacent GHz phase measurements to classify regions into their respective wrap counts. We validate this method in simulation and experimentally, and demonstrate precise depth sensing using centimeter wave modulation that is robust to surface texture and ambient light. Compared to existing analog demodulation methods, the proposed system outperforms all of them across all tested scenarios.Keywords
Funding Information
- NSF Graduate Research Fellowship
- NSF CAREER Award (2047359)
This publication has 91 references indexed in Scilit:
- Low-budget transient imaging using photonic mixer devicesACM Transactions on Graphics, 2013
- Ford Campus vision and lidar data setThe International Journal of Robotics Research, 2011
- Mapping the world in 3DNature Photonics, 2010
- Review of Stereo Vision Algorithms: From Software to HardwareInternational Journal of Optomechatronics, 2008
- Solid-state time-of-flight range cameraIEEE Journal of Quantum Electronics, 2001
- Depth from defocus: A spatial domain approachInternational Journal of Computer Vision, 1994
- Active stereo: integrating disparity, vergence, focus, aperture and calibration for surface estimationIEEE Transactions on Pattern Analysis and Machine Intelligence, 1993
- Phase noise limited resolution of coherent LIDAR using widely tunable laser diodesElectronics Letters, 1992
- A high-frequency sinusoidal phase-modulation interferometer using an electro-optic modulator: Development and evaluationReview of Scientific Instruments, 1991
- Distance measurement by means of a light ray modulated at a microwave frequencyJournal of Scientific Instruments, 1961