Improving parallel I/O performance with data layout awareness

Yong Chen, Xian He Sun, Rajeev Thakur, Huaiming Song, Hui Jin

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

16 Scopus citations

Abstract

Parallel applications can benefit greatly from massive computational capability, but their performance suffers from large latency of I/O accesses. The poor I/O performance has been attributed as a critical cause of the low sustained performance of parallel computing systems. In this study, we propose a data layout-aware optimization strategy to promote a better integration of the parallel I/O middleware and parallel file systems, two major components of the current parallel I/O systems, and to improve the data access performance. We explore the layout-aware optimization in both independent I/O and collective I/O, two primary forms of I/O in parallel applications. We illustrate that the layout-aware I/O optimization could improve the performance of current parallel I/O strategy effectively. The experimental results verify that the proposed strategy could improve parallel I/O performance by nearly 40% on average. The proposed layout-aware parallel I/O has a promising potential in improving the I/O performance of parallel systems.

Original languageEnglish
Title of host publicationProceedings - 2010 IEEE International Conference on Cluster Computing, Cluster 2010
Pages302-311
Number of pages10
DOIs
StatePublished - 2010

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
ISSN (Print)1552-5244

Keywords

  • Collective I/O
  • Data access optimization
  • Data layout
  • I/O performance
  • Independent I/O
  • Parallel I/O
  • Parallel I/O middleware
  • Parallel file systems

Fingerprint

Dive into the research topics of 'Improving parallel I/O performance with data layout awareness'. Together they form a unique fingerprint.

Cite this