Accelerating Large-Scale Graph Neural Network Training on Crossbar Diet

Journal Article (Journal Article)

Resistive random-access memory (ReRAM)-based manycore architectures enable acceleration of graph neural network (GNN) inference and training. GNNs exhibit characteristics of both DNNs and graph analytics. Hence, GNN training/inferencing on ReRAM-based manycore architectures give rise to both computation and on-chip communication challenges. In this work, we leverage model pruning and efficient graph storage to reduce the computation and communication bottlenecks associated with GNN training on ReRAM-based manycore accelerators. However, traditional pruning techniques are either targeted for inferencing only, or they are not crossbar-aware. In this work, we propose a GNN pruning technique called DietGNN. DietGNN is a crossbar-aware pruning technique that achieves high accuracy training and enables energy, area, and storage efficient computing on ReRAM-based manycore platforms. The DietGNN pruned model can be trained from scratch without any noticeable accuracy loss. Our experimental results show that when mapped on to a ReRAM-based manycore architecture, DietGNN can reduce the number of crossbars by over 90% and accelerate GNN training by ${\sim }{2}.{7}{\times }$ compared to its unpruned counterpart. In addition, DietGNN reduces energy consumption by more than ${\sim }{3}.{5}{\times }$ compared to the unpruned counterpart.

Full Text

Duke Authors

Cited Authors

  • Ogbogu, C; Arka, AI; Joardar, BK; Doppa, JR; Li, H; Chakrabarty, K; Pande, PP

Published Date

  • November 1, 2022

Published In

Volume / Issue

  • 41 / 11

Start / End Page

  • 3626 - 3637

Electronic International Standard Serial Number (EISSN)

  • 1937-4151

International Standard Serial Number (ISSN)

  • 0278-0070

Digital Object Identifier (DOI)

  • 10.1109/TCAD.2022.3197342

Citation Source

  • Scopus