Contention-aware resource scheduling for burst buffer systems

Weihao Liang, Yong Chen, Jialin Liu, Hong An

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Many scientific applications in critical areas are becoming more and more data-intensive. As the data volume continues to grow, the data movement between storage and compute nodes has turned into a crucial performance bottleneck for many data-intensive applications. Burst buffer provides a promising solution for these applications by absorbing bursty I/O traffic to let applications return to computing phase quickly. However, the resource allocation policy for burst buffer is understudied, and the existing strategies may cause severe I/O contention when a large number of I/O-intensive jobs access the burst buffer system concurrently. In this study, based on the theoretic analysis of I/O model, we present a contention-aware resource scheduling (CARS) strategy that manages the burst buffer resource to coordinate concurrent jobs. Extensive experiments have been conducted and the results have demonstrated that the proposed CARS design outperforms the existing allocation strategies and improves both job performance and system utilization.

Original languageEnglish
Title of host publication47th International Conference on Parallel Processing, ICPP 2018
Subtitle of host publicationWorkshop Proceedings
PublisherAssociation for Computing Machinery
ISBN (Print)9781450365239
DOIs
StatePublished - Aug 13 2018
Event47th International Conference on Parallel Processing, ICPP 2018 - Eugene, United States
Duration: Aug 13 2018Aug 16 2018

Publication series

NameACM International Conference Proceeding Series

Conference

Conference47th International Conference on Parallel Processing, ICPP 2018
CountryUnited States
CityEugene
Period08/13/1808/16/18

Keywords

  • Burst buffer
  • I/O contention
  • I/O system
  • Modeling
  • Resource management
  • Runtime system
  • Scheduling algorithm

Fingerprint Dive into the research topics of 'Contention-aware resource scheduling for burst buffer systems'. Together they form a unique fingerprint.

  • Cite this

    Liang, W., Chen, Y., Liu, J., & An, H. (2018). Contention-aware resource scheduling for burst buffer systems. In 47th International Conference on Parallel Processing, ICPP 2018: Workshop Proceedings [32] (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3229710.3229718