Traffic vehicle cognition in severe weather based on radar and infrared thermal camera fusion

Abstract

The accuracy and robustness of vehicle cognition in severe weather has always been the focus and difficulty of intelligent vehicle environment perception. This paper proposed a vehicle cognition method based on radar and infrared thermal camera information fusion in severe weather. The fusion of radar and infrared camera can greatly enrich the completeness of vehicle cognitive information. Firstly, an attention mechanism based on radar guidance information was proposed to extract vehicle ROI. Unlike the traditional ROI extraction method, this method is not disturbed by the environment and does not need complicated calculations, which can quickly and accurately extract ROI for vehicles. Secondly, based on ROI information, the infrared thermal image was reconstructed and enhanced, which is of great significance to improve the accuracy of vehicle detection for deep learning. Finally, we propose a vehicle depth estimation method based on pixel regression and use multi-scale cognitive information to fuse radar and image targets. The fusion method considering depth information can reduce target confusion and improve fusion robustness and accuracy, especially when vehicles are adjacent to each other. The experimental results show that the vehicle detection accuracy of this method is 95.2% and the vehicle detection speed is 37 Fps, which effectively improves the performance of vehicle detection in severe weather.