ISAAC
Top Cited Papers
- 18 June 2016
- journal article
- conference paper
- Published by Association for Computing Machinery (ACM) in ACM SIGARCH Computer Architecture News
- Vol. 44 (3), 14-26
- https://doi.org/10.1145/3007787.3001139
Abstract
A number of recent efforts have attempted to design accelerators for popular machine learning algorithms, such as those involving convolutional and deep neural networks (CNNs and DNNs). These algorithms typically involve a large number of multiply-accumulate (dot-product) operations. A recent project, DaDianNao, adopts a near data processing approach, where a specialized neural functional unit performs all the digital arithmetic operations and receives input weights from adjacent eDRAM banks. This work explores an in-situ processing approach, where memristor crossbar arrays not only store input weights, but are also used to perform dot-product operations in an analog manner. While the use of crossbar memory as an analog dot-product engine is well known, no prior work has designed or characterized a full-fledged accelerator based on crossbars. In particular, our work makes the following contributions: (i) We design a pipelined architecture, with some crossbars dedicated for each neural network layer, and eDRAM buffers that aggregate data between pipeline stages. (ii) We define new data encoding techniques that are amenable to analog computations and that can reduce the high overheads of analog-to-digital conversion (ADC). (iii) We define the many supporting digital components required in an analog CNN accelerator and carry out a design space exploration to identify the best balance of memristor storage/compute, ADCs, and eDRAM storage on a chip. On a suite of CNN and DNN workloads, the proposed ISAAC architecture yields improvements of 14.8×, 5.5×, and 7.5× in throughput, energy, and computational density (respectively), relative to the state-of-the-art DaDianNao architecture.Keywords
This publication has 71 references indexed in Scilit:
- Pattern classification by memristive crossbar circuits using ex situ and in situ trainingNature Communications, 2013
- A Functional Hybrid Memristor Crossbar-Array/CMOS System for Data Storage and Neuromorphic ApplicationsNano Letters, 2011
- Experimental demonstration of associative memory with memristive neural networksNeural Networks, 2010
- Writing to and reading from a nano-scale crossbar memory based on memristorsNanotechnology, 2009
- The missing memristor foundNature, 2008
- A Convolutional Neural Network Tolerant of Synaptic Faults for Low-Power Analog HardwareLecture Notes in Computer Science, 2006
- Charge-mode parallel architecture for vector-matrix multiplicationIEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 2001
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998
- Application of the ANNA neural network chip to high-speed character recognitionIEEE Transactions on Neural Networks, 1992
- An analog neural network processor with programmable topologyIEEE Journal of Solid-State Circuits, 1991