Automated Performance Modeling of HPC Applications Using Machine Learning

Jingwei Sun, Guangzhong Sun, Shiyan Zhan, Jiepeng Zhang, Yong Chen

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Automated performance modeling and performance prediction of parallel programs are highly valuable in many use cases, such as in guiding task management and job scheduling, offering insights of application behaviors, and assisting resource requirement estimation. The performance of parallel programs is affected by numerous factors, including but not limited to hardware, applications, algorithms, and input parameters, thus an accurate performance prediction is often a challenging and daunting task. In this article, we focus on automatically predicting the execution time of parallel programs (more specifically, MPI programs) with different inputs, at different scales, and without domain knowledge. We model the correlation between the execution time and domain-independent runtime features. These features include values of variables, counters of branches, loops, and MPI communications. Through automatically instrumenting an MPI program, each execution of the program will output a feature vector and its corresponding execution time. After collecting data from executions with different inputs, a random forest machine learning approach is used to build an empirical performance model, which can predict the execution time of the program given a new input. A transfer learning method is used to reuse an existing performance model and improve the prediction accuracy on a new platform that lacks historical execution data. Our experiments and analyses of three parallel applications, Graph500, GalaxSee, and SMG2000, on three different systems confirm that our method performs well, with less than 20 percent prediction error on average.

Original languageEnglish
Article number8956059
Pages (from-to)749-763
Number of pages15
JournalIEEE Transactions on Computers
Volume69
Issue number5
DOIs
StatePublished - May 1 2020

Keywords

  • Parallel computing
  • machine learning
  • model transferring
  • performance modeling

Fingerprint

Dive into the research topics of 'Automated Performance Modeling of HPC Applications Using Machine Learning'. Together they form a unique fingerprint.

Cite this