Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation
- 1 October 2019
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4176-4185
- https://doi.org/10.1109/iccvw.2019.00513
Abstract
Convolutional Neural Networks (CNN) have become state-of-the-art in the field of image classification. However, not everything is understood about their inner representations. This paper tackles the interpretability and explainability of the predictions of CNNs for multi-class classification problems. Specifically, we propose a novel visualization method of pixel-wise input attribution called Softmax-Gradient Layer-wise Relevance Propagation (SGLRP). The proposed model is a class discriminate extension to Deep Taylor Decomposition (DTD) using the gradient of softmax to back propagate the relevance of the output probability to the input image. Through qualitative and quantitative analysis, we demonstrate that SGLRP can successfully localize and attribute the regions on input images which contribute to a target object's classification. We show that the proposed method excels at discriminating the target objects class from the other possible objects in the images. We confirm that SGLRP performs better than existing Layer-wise Relevance Propagation (LRP) based methods and can help in the understanding of the decision process of CNNs.Keywords
This publication has 30 references indexed in Scilit:
- Visualizing Deep Convolutional Neural Networks Using Natural Pre-imagesInternational Journal of Computer Vision, 2016
- Self-taught object localization with deep networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet ClassificationPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- Predicting effects of noncoding variants with deep learning–based sequence modelNature Methods, 2015
- Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learningNature Biotechnology, 2015
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance PropagationPLOS ONE, 2015
- DevNet: A Deep Event Network for multimedia event detection and evidence recountingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998