An Energy-Efficient Inference Method in Convolutional Neural Networks Based on Dynamic Adjustment of the Pruning Level
- 1 August 2021
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Design Automation of Electronic Systems
- Vol. 26 (6), 1-20
- https://doi.org/10.1145/3460972
Abstract
In this article, we present a low-energy inference method for convolutional neural networks in image classification applications. The lower energy consumption is achieved by using a highly pruned (lower-energy) network if the resulting network can provide a correct output. More specifically, the proposed inference method makes use of two pruned neural networks (NNs), namely mildly and aggressively pruned networks, which are both designed offline. In the system, a third NN makes use of the input data for the online selection of the appropriate pruned network. The third network, for its feature extraction, employs the same convolutional layers as those of the aggressively pruned NN, thereby reducing the overhead of the online management. There is some accuracy loss induced by the proposed method where, for a given level of accuracy, the energy gain of the proposed method is considerably larger than the case of employing any one pruning level. The proposed method is independent of both the pruning method and the network architecture. The efficacy of the proposed inference method is assessed on Eyeriss hardware accelerator platform for some of the state-of-the-art NN architectures. Our studies show that this method may provide, on average, 70% energy reduction compared to the original NN at the cost of about 3% accuracy loss on the CIFAR-10 dataset.Keywords
This publication has 16 references indexed in Scilit:
- In-Datacenter Performance Analysis of a Tensor Processing UnitPublished by Association for Computing Machinery (ACM) ,2017
- Real-Time Meets Approximate ComputingPublished by Association for Computing Machinery (ACM) ,2017
- Notice of Violation of IEEE Publication Principles: Dissipativity-Based Fuzzy Integral Sliding Mode Control of Continuous-Time T-S Fuzzy SystemsIEEE Transactions on Fuzzy Systems, 2017
- Energy-Efficient and Improved Image Recognition with Conditional Deep LearningACM Journal on Emerging Technologies in Computing Systems, 2017
- Dermatologist-level classification of skin cancer with deep neural networksNature, 2017
- Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural NetworksIEEE Journal of Solid-State Circuits, 2016
- XNOR-Net: ImageNet Classification Using Binary Convolutional Neural NetworksPublished by Springer Science and Business Media LLC ,2016
- DeepDriving: Learning Affordance for Direct Perception in Autonomous DrivingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- AxNNPublished by Association for Computing Machinery (ACM) ,2014
- Peter Huttenlocher (1931–2013)Nature, 2013