Thread batching for high-performance energy-efficient GPU memory design


Journal Article

© 2019 Association for Computing Machinery. Massive multi-threading in GPU imposes tremendous pressure on memory subsystems. Due to rapid growth in thread-level parallelism of GPU and slowly improved peak memory bandwidth, memory becomes a bottleneck of GPU’s performance and energy efficiency. In this article, we propose an integrated architectural scheme to optimize the memory accesses and therefore boost the performance and energy efficiency of GPU. First, we propose a thread batch enabled memory partitioning (TEMP) to improve GPU memory access parallelism. In particular, TEMP groups multiple thread blocks that share the same set of pages into a thread batch and applies a page coloring mechanism to bound each stream multiprocessor (SM) to the dedicated memory banks. After that, TEMP dispatches the thread batch to an SM to ensure high-parallel memory-access streaming from the different thread blocks. Second, a thread batch-aware scheduling (TBAS) scheme is introduced to improve the GPU memory access locality and to reduce the contention on memory controllers and interconnection networks. Experimental results show that the integration of TEMP and TBAS can achieve up to 10.3% performance improvement and 11.3% DRAM energy reduction across diverse GPU applications. We also evaluate the performance interference of the mixed CPU+GPU workloads when they are run on a heterogeneous system that employs our proposed schemes. Our results show that a simple solution can effectively ensure the efficient execution of both GPU and CPU applications.

Full Text

Duke Authors

Cited Authors

  • Li, B; Mao, M; Liu, X; Liu, T; Liu, Z; Wen, W; Chen, Y; Li, HH

Published Date

  • December 1, 2019

Published In

Volume / Issue

  • 15 / 4

Electronic International Standard Serial Number (EISSN)

  • 1550-4840

International Standard Serial Number (ISSN)

  • 1550-4832

Digital Object Identifier (DOI)

  • 10.1145/3330152

Citation Source

  • Scopus