Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars
Top Cited Papers
- 1 June 2018
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5419-5427
- https://doi.org/10.1109/cvpr.2018.00568
Abstract
Event cameras are bio-inspired vision sensors that naturally capture the dynamics of a scene, filtering out redundant information. This paper presents a deep neural network approach that unlocks the potential of event cameras on a challenging motion-estimation task: prediction of a vehicle's steering angle. To make the best out of this sensor-algorithm combination, we adapt state-of-the-art convolutional architectures to the output of event sensors and extensively evaluate the performance of our approach on a publicly available large scale event-camera dataset (~1000 km). We present qualitative and quantitative explanations of why event cameras allow robust steering prediction even in cases where traditional cameras fail, e.g. challenging illumination conditions and fast motion. Finally, we demonstrate the advantages of leveraging transfer learning from traditional to event-based vision, and show that our approach outperforms state-of-the-art algorithms based on standard cameras.Keywords
This publication has 13 references indexed in Scilit:
- EVO: A Geometric Approach to Event-Based 6-DOF Parallel Tracking and Mapping in Real TimeIEEE Robotics and Automation Letters, 2016
- HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Learning Deep Features for Discriminative LocalizationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- HFirst: A Temporal Approach to Object RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- A 240 × 180 130 dB 3 µs Latency Global Shutter Spatiotemporal Vision SensorIEEE Journal of Solid-State Circuits, 2014
- A spiking neural network architecture for visual motion estimationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Mapping from Frame-Driven to Frame-Free Event-Driven Vision Systems by Low-Rate Rate Coding and Coincidence Processing--Application to Feedforward ConvNetsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision SensorIEEE Journal of Solid-State Circuits, 2008
- Simple statistical gradient-following algorithms for connectionist reinforcement learningMachine Learning, 1992