Iteration based collective I/O strategy for Parallel I/O systems

Zhixiang Wang, Xuanhua Shi, Hai Jin, Song Wu, Yong Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Scopus citations

Abstract

MPI collective I/O is a widely used I/O method that helps data-intensive scientific applications gain better I/O performance. However, it has been observed that existing collective I/O strategies do not perform well due to the access contention problem. Existing collective I/O optimization strategies mainly focus on the I/O phase efficiency and ignore the shuffle cost that may limit the potential of their performance improvement. We observe that as the size of I/O becomes larger, one I/O operation from the upper application would be separated into several iterations to complete. So, I/O requests in each file domain do not necessarily issue to the parallel file system simultaneously unless they are carried out within the same iteration step. Based on that observation, this paper proposes a new collective I/O strategy that reorganizes I/O requests within each file domain instead of coordinating requests across file domains, such that we can eliminate access contentions without introducing extra shuffle cost between aggregators and computing processes. Using benchmark workloads IOR, we evaluate our new strategy and compare with the conventional one. The proposed strategy achieves up to 47%-63% I/O bandwidth improvement compared to the existing ROMIO collective I/O strategy.

Original languageEnglish
Title of host publicationProceedings - 14th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2014
PublisherIEEE Computer Society
Pages287-294
Number of pages8
ISBN (Print)9781479927838
DOIs
StatePublished - 2014
Event14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2014 - Chicago, IL, United States
Duration: May 26 2014May 29 2014

Publication series

NameProceedings - 14th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, CCGrid 2014

Conference

Conference14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2014
CountryUnited States
CityChicago, IL
Period05/26/1405/29/14

Keywords

  • access contention
  • collective I/O
  • iteration
  • parallel system

Fingerprint Dive into the research topics of 'Iteration based collective I/O strategy for Parallel I/O systems'. Together they form a unique fingerprint.

Cite this