Contention-aware resource scheduling for burst buffer systems

Many scientific applications in critical areas are becoming more and more data-intensive. As the data volume continues to grow, the data movement between storage and compute nodes has turned into a crucial performance bottleneck for many data-intensive applications. Burst buffer provides a promising solution for these applications by absorbing bursty I/O traffic to let applications return to computing phase quickly. However, the resource allocation policy for burst buffer is understudied, and the existing strategies may cause severe I/O contention when a large number of I/O-intensive jobs access the burst buffer system concurrently. In this study, based on the theoretic analysis of I/O model, we present a contention-aware resource scheduling (CARS) strategy that manages the burst buffer resource to coordinate concurrent jobs. Extensive experiments have been conducted and the results have demonstrated that the proposed CARS design outperforms the existing allocation strategies and improves both job performance and system utilization.