Automated performance modeling based on runtime feature detection and machine learning

Automated performance modeling and performance prediction of parallel programs are highly valuable in many use cases, such as in guiding task management and job scheduling, offering insights of application behaviors, assisting resource requirement estimation, etc. The performance of parallel programs is affected by numerous factors, including but not limited to hardware, system software, applications, algorithms, and input parameters, thus an accurate performance prediction is often a challenging and daunting task. In this study, we focus on automatically predicting the execution time of parallel programs (more specifically, MPI programs) with different inputs, at different scale, and without domain knowledge. We model the correlation between the execution time and domain-independent runtime features. These features include values of variables, counters of branches, loops, and MPI communications. Through automatically instrumenting an MPI program, each execution of the program will output a feature vector and its corresponding execution time. After collecting data from executions with different inputs, a random forest machine learning approach is used to build an empirical performance model, which can predict the execution time of the program with a new input. Our experiments and analyses of three parallel programs, Graph500, GalaxSee and SMG2000, on three different systems show that our method performs well, with less than 20% error in predictions on average.