Log-Assisted Straggler-Aware I/O Scheduler for High-End Computing

Neda Tavakoli, Dong Dai, Yong Chen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

7 Scopus citations

Abstract

Object-based parallel file systems have emerged as promising storage solutions for high-end computing (HEC) systems. Despite the fact that object storage provides a flexible interface, scheduling highly concurrent I/O requests that access a large number of objects still remains as a challenging problem, especially in the case when stragglers (storage servers that are significantly slower than others) exist in the system. An efficient I/O scheduler needs to avoid possible stragglers to achieve low latency and high throughput. In this paper, we introduce a log-assisted straggler-aware I/O scheduling to mitigate the impact of storage server stragglers. The contribution of this study is threefold. First, we introduce a client-side, log-assisted, straggler-aware I/O scheduler architecture to tackle the storage straggler issue in HEC systems. Second, we present two scheduling algorithms that can make efficient decision on scheduling I/Os while avoiding stragglers based on such an architecture. Third, we evaluate the proposed I/O scheduler using simulations. The simulation results have confirmed the promise of the newly introduced log-assisted straggler-aware I/O scheduler in large-scale HEC systems.

Original languageEnglish
Title of host publicationProceedings - 45th International Conference on Parallel Processing Workshops, ICPPW 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages181-189
Number of pages9
ISBN (Electronic)9781509028252
DOIs
StatePublished - Sep 23 2016
Event45th International Conference on Parallel Processing Workshops, ICPPW 2016 - Philadelphia, United States
Duration: Aug 16 2016Aug 19 2016

Publication series

NameProceedings of the International Conference on Parallel Processing Workshops
Volume2016-September
ISSN (Print)1530-2016

Conference

Conference45th International Conference on Parallel Processing Workshops, ICPPW 2016
CountryUnited States
CityPhiladelphia
Period08/16/1608/19/16

Keywords

  • High-end computing
  • I/O scheduler
  • Parallel file systems

Fingerprint Dive into the research topics of 'Log-Assisted Straggler-Aware I/O Scheduler for High-End Computing'. Together they form a unique fingerprint.

Cite this