Skip to main content

ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs

Publication ,  Journal Article
Joardar, BK; Doppa, JR; Li, H; Chakrabarty, K; Pande, PP
Published in: IEEE Transactions on Emerging Topics in Computing
April 1, 2023

Training machine learning (ML) models at the edge (on-chip training on end user devices) can address many pressing challenges including data privacy/security, increase the accessibility of ML applications to different parts of the world by reducing the dependence on the communication fabric and the cloud infrastructure, and meet the real-time requirements of AR/VR applications. However, existing edge platforms do not have sufficient computing capabilities to support complex ML tasks such as training large CNNs. ReRAM-based architectures offer high-performance yet energy efficient computing platforms for on-chip CNN training/inferencing. However, ReRAM-based architectures are not scalable with the size of the CNN. Larger CNNs have more weights, which requires more ReRAM cells that cannot be integrated in a single chip. Moreover, training larger CNNs on-chip will require higher power, which cannot be afforded by these smaller devices. Pruning is an effective way to solve this problem. However, existing pruning techniques are either targeted for inferencing only, or they are not crossbar-aware. This leads to sub-optimal hardware savings and performance benefits for CNN training on ReRAM-based architectures. In this paper, we address this problem by proposing a novel crossbar-aware pruning strategy, referred as ReaLPrune, which can prune more than 90% of CNN weights. The pruned model can be trained from scratch without any accuracy loss. Experimental results indicate that ReaLPrune reduces hardware requirements by 77.2% and accelerates CNN training by ∼20× compared to unpruned CNNs. ReaLPrune also outperforms other crossbar-aware pruning techniques in terms of both performance and hardware savings. In addition, ReaLPrune is equally effective for diverse datasets and more complex CNNs.

Duke Scholars

Published In

IEEE Transactions on Emerging Topics in Computing

DOI

EISSN

2168-6750

Publication Date

April 1, 2023

Volume

11

Issue

2

Start / End Page

303 / 317

Related Subject Headings

  • 46 Information and computing sciences
  • 0906 Electrical and Electronic Engineering
  • 0806 Information Systems
  • 0805 Distributed Computing
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Joardar, B. K., Doppa, J. R., Li, H., Chakrabarty, K., & Pande, P. P. (2023). ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs. IEEE Transactions on Emerging Topics in Computing, 11(2), 303–317. https://doi.org/10.1109/TETC.2022.3223630
Joardar, B. K., J. R. Doppa, H. Li, K. Chakrabarty, and P. P. Pande. “ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs.” IEEE Transactions on Emerging Topics in Computing 11, no. 2 (April 1, 2023): 303–17. https://doi.org/10.1109/TETC.2022.3223630.
Joardar BK, Doppa JR, Li H, Chakrabarty K, Pande PP. ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs. IEEE Transactions on Emerging Topics in Computing. 2023 Apr 1;11(2):303–17.
Joardar, B. K., et al. “ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs.” IEEE Transactions on Emerging Topics in Computing, vol. 11, no. 2, Apr. 2023, pp. 303–17. Scopus, doi:10.1109/TETC.2022.3223630.
Joardar BK, Doppa JR, Li H, Chakrabarty K, Pande PP. ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs. IEEE Transactions on Emerging Topics in Computing. 2023 Apr 1;11(2):303–317.

Published In

IEEE Transactions on Emerging Topics in Computing

DOI

EISSN

2168-6750

Publication Date

April 1, 2023

Volume

11

Issue

2

Start / End Page

303 / 317

Related Subject Headings

  • 46 Information and computing sciences
  • 0906 Electrical and Electronic Engineering
  • 0806 Information Systems
  • 0805 Distributed Computing