HyperLink   Accelerating Reduction and Scan Using Tensor Core Units
Publication Year:
  Abdul Dakkak, Cheng Li, Jinjun Xiong, Wen-mei Hwu
  ICS 2019: International Conference on Supercomputing, June 26-28, Phoenix AZ

Driven by deep learning, there has been a surge of specialized processors for matrix multiplication, referred to as Tensor Core Units (TCUs). These TCUs are capable of performing matrix multiplications on small matrices (usually 4×4 or 16×16) to accelerate HPC and deep learning workloads. Although TCUs are prevalent and promise increase in performance and/or energy efficiency, they suffer from over specialization as only matrix multiplication on small matrices is supported. In this paper we express both reduction and scan in terms of matrix multiplication operations and map them onto TCUs. To our knowledge, this paper is the first to try to broaden the class of algorithms expressible as TCU operations and is the first to show benefits of this mapping in terms of: program simplicity, efficiency, and performance. We implemented the reduction and scan algorithms using NVIDIAs V100 TCUs and achieved 89%98% of peak memory copy bandwidth. Our results are orders of magnitude faster (up to 100× for reduction and 3× for scan) than state-of-the-art methods for small segment sizes (common in HPC and deep learning applications). Our implementation achieves this speedup while decreasing the power consumption by up to 22% for reduction and 16% for scan.

author = {Dakkak, Abdul and Li, Cheng and Xiong, Jinjun and Gelado, Isaac and Hwu, Wen-mei},
title = {Accelerating Reduction and Scan Using Tensor Core Units},
booktitle = {Proceedings of the ACM International Conference on Supercomputing},
series = {ICS '19},
year = {2019},
isbn = {978-1-4503-6079-1},
location = {Phoenix, Arizona},
pages = {46--57},
numpages = {12},
url = {http://doi.acm.org/10.1145/3330345.3331057},
doi = {10.1145/3330345.3331057},
acmid = {3331057},
publisher = {ACM},
address = {New York, NY, USA},