Attention-Based Multi-Scale Convolutional Neural Network (A+MCNN) for Multi-Class Classification in Road Images
Open Access
- 29 July 2021
- Vol. 21 (15), 5137
- https://doi.org/10.3390/s21155137
Abstract
Automated pavement distress recognition is a key step in smart infrastructure assessment. Advances in deep learning and computer vision have improved the automated recognition of pavement distresses in road surface images. This task remains challenging due to the high variation of defects in shapes and sizes, demanding a better incorporation of contextual information into deep networks. In this paper, we show that an attention-based multi-scale convolutional neural network (A+MCNN) improves the automated classification of common distress and non-distress objects in pavement images by (i) encoding contextual information through multi-scale input tiles and (ii) employing a mid-fusion approach with an attention module for heterogeneous image contexts from different input scales. A+MCNN is trained and tested with four distress classes (crack, crack seal, patch, pothole), five non-distress classes (joint, marker, manhole cover, curbing, shoulder), and two pavement classes (asphalt, concrete). A+MCNN is compared with four deep classifiers that are widely used in transportation applications and a generic CNN classifier (as the control model). The results show that A+MCNN consistently outperforms the baselines by 1∼26% on average in terms of the F-score. A comprehensive discussion is also presented regarding how these classifiers perform differently on different road objects, which has been rarely addressed in the existing literature.Keywords
This publication has 43 references indexed in Scilit:
- Improvement of Crack-Detection Accuracy Using a Novel Crack Defragmentation Technique in Image-Based Road AssessmentJournal of Computing in Civil Engineering, 2016
- Look and Think Twice: Capturing Top-Down Visual Attention with Feedback Convolutional Neural NetworksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Semantic Image Segmentation via Deep Parsing NetworkPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional ArchitecturePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Learning Deconvolution Network for Semantic SegmentationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- AttentionNet: Aggregating Weak Directions for Accurate Object DetectionPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- U-Net: Convolutional Networks for Biomedical Image SegmentationPublished by Springer Science and Business Media LLC ,2015
- Crack Recognition and Segmentation Using Morphological Image-Processing Techniques for Flexible PavementsTransportation Research Record: Journal of the Transportation Research Board, 2015
- Pothole detection in asphalt pavement imagesAdvanced Engineering Informatics, 2011
- A Novel LBP Based Methods for Pavement Crack DetectionJournal of Pattern Recognition Research, 2010