DyNNamic: Dynamically Reshaping, High Data-Reuse Accelerator for Compact DNNs

Journal Article (Journal Article)

Convolutional layers dominate the computation and energy costs of Deep Neural Network (DNN) inference. Recent algorithmic works attempt to reduce these bottlenecks via compact DNN structures and model compression. Likewise, state-of-the-art accelerator designs leverage spatiotemporal characteristics of convolutional layers to reduce data movement overhead and improve throughput. Although both are independently effective at reducing latency and energy costs, combining these approaches does not guarantee cumulative improvements due to inefficient mapping. This inefficiency can be attributed to (1) inflexibility of underlying hardware and (2) inherent reduction of data-reuse opportunities of compact DNN structures. To address these issues, we propose a dynamically reshaping, high data-reuse PE array accelerator, namely DyNNamic. DyNNamic leverages kernel-wise filter decomposition to partition the convolution operation into two compact stages: Shared Kernels Convolution (SKC) and Weighted Accumulation (WA). Because both stages have vastly different dimensions, DyNNamic reshapes its PE array to effectively map the algorithm to the architecture. The architecture then exploits data-reuse opportunities created by the SKC stage, further reducing data movement with negligible overhead. We evaluate our approach on various representative networks and compare against state-of-the-art accelerators. On average, DyNNamic outperforms DianNao by $8.4\times$ and $12.3\times$ in terms of inference energy and latency, respectively.

Full Text

Duke Authors

Cited Authors

  • Hanson, E; Li, S; Qian, X; Li, H; Chen, Y

Published Date

  • January 1, 2022

Published In

Electronic International Standard Serial Number (EISSN)

  • 1557-9956

International Standard Serial Number (ISSN)

  • 0018-9340

Digital Object Identifier (DOI)

  • 10.1109/TC.2022.3184272

Citation Source

  • Scopus