Performance model-directed data sieving for high-performance I/O

Yong Chen, Yin Lu, Prathamesh Amritkar, Rajeev Thakur, Yu Zhuang

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Many scientific computing applications and engineering simulations exhibit noncontiguous I/O access patterns. Data sieving is an important technique to improve the performance of noncontiguous I/O accesses by combining small and noncontiguous requests into a large and contiguous request. It has been proven effective even though more data are potentially accessed than demanded. In this study, we propose a new data sieving approach namely performance model-directed data sieving, or PMD data sieving in short. It improves the existing data sieving approach from two aspects: (1) dynamically determines when it is beneficial to perform data sieving; and (2) dynamically determines how to perform data sieving if beneficial. It improves the performance of the existing data sieving approach considerably and reduces the memory consumption as verified by both theoretical analysis and experimental results. Given the importance of supporting noncontiguous accesses effectively and reducing the memory pressure in a large-scale system, the proposed PMD data sieving approach in this research holds a great promise and will have an impact on high-performance I/O systems.

Original languageEnglish
Pages (from-to)2066-2090
Number of pages25
JournalJournal of Supercomputing
Volume71
Issue number6
DOIs
StatePublished - Jun 29 2015

Keywords

  • Data sieving
  • High-performance computing
  • Libraries
  • Parallel I/O
  • Parallel file systems
  • Runtime systems

Fingerprint Dive into the research topics of 'Performance model-directed data sieving for high-performance I/O'. Together they form a unique fingerprint.

Cite this