HAM: Hotspot-aware manager for improving communications with 3D-stacked memory

Xi Wang, Antonino Tumeo, John D. Leidel, Jie Li, Yong Chen

Research output: Contribution to journalArticlepeer-review

Abstract

Emerging High-Performance Computing (HPC) workloads, such as graph analytics, machine learning, and big data science, are data-intensive. Data-intensive workloads usually present fine-grained memory accesses with limited or no data locality, and thus incur frequent cache misses and low utilization of memory bandwidth. 3D-stacked memory devices such as Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM) can provide significantly higher bandwidth than conventional memory modules. However, the traditional interfaces and optimization methods for JEDEC DDR devices do not allow to fully exploit the potential performance of 3D-stacked memory with the massive amount of irregular memory accesses of data-intensive applications. In this article, we propose a novel Hotspot-Aware Manager (HAM) infrastructure for 3D-stacked memory devices capable of optimizing memory access streams via request aggregation, hotspot detection, and in-memory prefetching. We present the HAM design and implementation, and simulate it on a system using RISC-V embedded cores with attached HMC devices. We extensively evaluate HAM with over 12 benchmarks and applications representing diverse irregular memory access patterns. The results show that, on average, HAM reduces redundant requests by 37.51 percent and increases the prefetch buffer hit rate by 4.2 times, compared to a baseline streaming prefetcher. On the selected benchmark set, HAM provides performance gains of 21.81 percent in average (up to 34.28 percent), and power savings of 35.07 percent over a standard 3D-stacked memory.

Original languageEnglish
Article number9381710
Pages (from-to)833-848
Number of pages16
JournalIEEE Transactions on Computers
Volume70
Issue number6
DOIs
StatePublished - Jun 1 2021

Keywords

  • 3D-stacked memory
  • HBM
  • HMC
  • Memory hotspot
  • coalescing
  • communications
  • prefetching

Fingerprint Dive into the research topics of 'HAM: Hotspot-aware manager for improving communications with 3D-stacked memory'. Together they form a unique fingerprint.

Cite this