VUNet: Dynamic Scene View Synthesis for Traversability Estimation Using an RGB Camera
- 23 January 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 4 (2), 2062-2069
- https://doi.org/10.1109/lra.2019.2894869
Abstract
We present VUNet, a novel view(VU) synthesis method for mobile robots in dynamic environments, and its application to the estimation of future traversability. Our method predicts future images for given virtual robot velocity commands using only RGB images at previous and current time steps. The future images result from applying two types of image changes to the previous and current images: 1) changes caused by different camera pose, and 2) changes due to the motion of the dynamic obstacles. We learn to predict these two types of changes disjointly using two novel network architectures, SNet and DNet. We combine SNet and DNet to synthesize future images that we pass to our previously presented method GONet [1] to estimate the traversable areas around the robot. Our quantitative and qualitative evaluation indicate that our approach for view synthesis predicts accurate future images in both static and dynamic environments. We also show that these virtual images can be used to estimate future traversability correctly. We apply our view synthesis-based traversabilityKeywords
Funding Information
- Toyota Research Institute
- The TOYOTA Central R&D Labs., INC. at Stanford University
This publication has 24 references indexed in Scilit:
- FlowNet: Learning Optical Flow with Convolutional NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional ArchitecturePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- U-Net: Convolutional Networks for Biomedical Image SegmentationPublished by Springer Science and Business Media LLC ,2015
- Learning to generate chairs with convolutional neural networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Traversability analysis for mobile robots in outdoor environments: A semi-supervised learning approach based on 3D-lidar dataPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- A depth space approach to human-robot collision avoidancePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2012
- Image Quality Assessment: From Error Visibility to Structural SimilarityIEEE Transactions on Image Processing, 2004
- Layered depth imagesPublished by Association for Computing Machinery (ACM) ,1998
- The dynamic window approach to collision avoidanceIEEE Robotics & Automation Magazine, 1997
- View morphingPublished by Association for Computing Machinery (ACM) ,1996