CARS: A contention-aware scheduler for efficient resource management of HPC storage systems

Weihao Liang, Yong Chen, Jialin Liu, Hong An

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Many scientific applications are becoming more and more data intensive. As the data volume continues to grow, the data movement between storage and compute nodes has turned into a crucial performance bottleneck for many data-intensive applications. Burst buffer provides a promising solution for these applications by absorbing bursty I/O traffic. However, the resource allocation and management strategies for burst buffer are not well studied. The existing bandwidth based strategies may cause severe I/O contention when a large number of I/O-intensive jobs access the burst buffer system concurrently. In this study, we present a contention-aware resource scheduling (CARS) strategy to manage the burst buffer resources and coordinate concurrent data-intensive jobs. The experimental results show that the proposed CARS framework outperforms the existing allocation strategies and improves both the job performance and the system utilization.

Original languageEnglish
Pages (from-to)25-34
Number of pages10
JournalParallel Computing
Volume87
DOIs
StatePublished - Sep 2019

Keywords

  • Burst buffer
  • I/O contention
  • I/O system
  • Modeling
  • Resource management
  • Scheduling algorithm

Fingerprint Dive into the research topics of 'CARS: A contention-aware scheduler for efficient resource management of HPC storage systems'. Together they form a unique fingerprint.

Cite this