Skip to main content

A Quantized Training Method to Enhance Accuracy of ReRAM-based Neuromorphic Systems

Publication ,  Conference
Yang, Q; Li, H; Wu, Q
Published in: Proceedings - IEEE International Symposium on Circuits and Systems
April 26, 2018

Deep neural networks (DNNs) are tremendously applied in artificial intelligence field. While the performance of DNNs is continuously improved by more complicated and deeper structures, the feasibility of deployment on embedded system remains as a critical problem. Neuromorphic system designs based on resistive random-access memory (ReRAM) provide an opportunity for power-efficient DNN employment. However, it encounters the challenge of limited programming resolution. A quantized training method is proposed in this paper to enhance the performance of neuromorphic systems based on ReRAM. Different from previous methods in which a dedicated regularization term is used in loss function to constrain parameter distribution, our quantized training method deals with training and quantization at the same time to alleviate the impact of limited parameter precision. Models with discrete parameters obtained after training can be directly mapped onto ReRAM devices. We implement experiments on image recognition tasks using multi-layer perceptron (MLP) and convolution neural network (CNN). The results verify that the quantized training method can approximate the accuracy of full-precision training, e.g., a two-layer MLP based on binary ReRAM only decreases the classification accuracy by 0.25% for MNIST dataset. In addition, we carefully investigate and present the importance of layer size under the ReRAM's low programming resolution, the different parameter resolution demands for convolution layer and fully connected layer, and system robustness to ReRAM variations after quantized training. The codes are available at https://github.com/qingyangqing/quantized-rram-net.

Duke Scholars

Published In

Proceedings - IEEE International Symposium on Circuits and Systems

DOI

ISSN

0271-4310

ISBN

9781538648810

Publication Date

April 26, 2018

Volume

2018-May
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Yang, Q., Li, H., & Wu, Q. (2018). A Quantized Training Method to Enhance Accuracy of ReRAM-based Neuromorphic Systems. In Proceedings - IEEE International Symposium on Circuits and Systems (Vol. 2018-May). https://doi.org/10.1109/ISCAS.2018.8351327
Yang, Q., H. Li, and Q. Wu. “A Quantized Training Method to Enhance Accuracy of ReRAM-based Neuromorphic Systems.” In Proceedings - IEEE International Symposium on Circuits and Systems, Vol. 2018-May, 2018. https://doi.org/10.1109/ISCAS.2018.8351327.
Yang Q, Li H, Wu Q. A Quantized Training Method to Enhance Accuracy of ReRAM-based Neuromorphic Systems. In: Proceedings - IEEE International Symposium on Circuits and Systems. 2018.
Yang, Q., et al. “A Quantized Training Method to Enhance Accuracy of ReRAM-based Neuromorphic Systems.” Proceedings - IEEE International Symposium on Circuits and Systems, vol. 2018-May, 2018. Scopus, doi:10.1109/ISCAS.2018.8351327.
Yang Q, Li H, Wu Q. A Quantized Training Method to Enhance Accuracy of ReRAM-based Neuromorphic Systems. Proceedings - IEEE International Symposium on Circuits and Systems. 2018.

Published In

Proceedings - IEEE International Symposium on Circuits and Systems

DOI

ISSN

0271-4310

ISBN

9781538648810

Publication Date

April 26, 2018

Volume

2018-May