Client-side straggler-aware I/O scheduler for object-based parallel file systems

Neda Tavakoli, Dong Dai, Yong Chen

Research output: Contribution to journalArticle

5 Scopus citations

Abstract

Object-based parallel file systems have emerged as promising storage solutions for high-performance computing (HPC) systems. Despite the fact that object storage provides a flexible interface, scheduling highly concurrent I/O requests that access a large number of objects still remains as a challenging problem, especially in the case when stragglers (storage servers that are significantly slower than others) exist in the system. An efficient I/O scheduler needs to avoid possible stragglers to achieve low latency and high throughput. In this paper, we introduce a log-assisted straggler-aware I/O scheduling to mitigate the impact of storage server stragglers. The contribution of this study is threefold. First, we introduce a client-side, log-assisted, straggler-aware I/O scheduler architecture to tackle the storage straggler issue in HPC systems. Second, we present three scheduling algorithms that can make efficient decision for scheduling I/Os while avoiding stragglers based on such an architecture. Third, we evaluate the proposed I/O scheduler using simulations, and the simulation results have confirmed the promise of the newly introduced straggler-aware I/O scheduler.

Original languageEnglish
Pages (from-to)3-18
Number of pages16
JournalParallel Computing
Volume82
DOIs
StatePublished - Feb 2019

Keywords

  • High performance computing
  • I/O scheduler
  • Parallel file systems
  • Straggler

Fingerprint Dive into the research topics of 'Client-side straggler-aware I/O scheduler for object-based parallel file systems'. Together they form a unique fingerprint.

  • Cite this