Collective prefetching for parallel I/O systems

Yong Chen, Philip C. Roth

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Data prefetching can be beneficial for improving parallel I/O system performance, but the amount of benefit depends on how efficiently and swiftly prefetches can be done. In this study, we propose a new prefetching strategy, called collective prefetching. The idea is to exploit the correlation among I/O accesses of multiple processes of a parallel application and carry out prefetches collectively, instead of the traditional strategy of carrying out prefetches by each process individually. The rationale behind this new collective prefetching strategy is that the concurrent processes of the same parallel application have strong correlation with respect to their I/O requests. We present the idea, initial design and implementation of the new collective prefetching strategy in this study. The preliminary experimental results show that this new collective prefetching strategy holds promise for improving parallel I/O performance.

Original languageEnglish
Title of host publicationProceedings of the 2010 5th Petascale Data Storage Workshop, PDSW'10, Held in Conjunction with SC'10 - The International Conference for High Performance Computing, Networking, Storage and Analysis
DOIs
StatePublished - 2010
Event2010 5th Petascale Data Storage Workshop, PDSW'10, Held in Conjunction with SC'10 - The International Conference for High Performance Computing, Networking, Storage and Analysis - New Orleans, LA, United States
Duration: Nov 15 2010Nov 15 2010

Publication series

NameProceedings of the 2010 5th Petascale Data Storage Workshop, PDSW'10, Held in Conjunction with SC'10 - The International Conference for High Performance Computing, Networking, Storage and Analysis

Conference

Conference2010 5th Petascale Data Storage Workshop, PDSW'10, Held in Conjunction with SC'10 - The International Conference for High Performance Computing, Networking, Storage and Analysis
Country/TerritoryUnited States
CityNew Orleans, LA
Period11/15/1011/15/10

Keywords

  • Collective prefetching
  • Exascale computing
  • High-performance computing
  • MPI-IO
  • Middleware
  • Parallel I/O
  • Parallel file systems
  • Performance
  • Prefetching
  • Storage

Cite this