Revealing applications' access pattern in collective I/O for cache management

Yin Lu, Yong Chen, Rob Latham, Yu Zhuang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

16 Scopus citations


Collective I/O is a critical I/O strategy on high-performance parallel computing systems that enables programmers to reveal parallel processes' I/O accesses collectively and makes possible for the parallel I/O middleware to carry out I/O requests in a highly efficient manner. Collective I/O has been proven as a core parallel I/O optimization technique. However, due to the collective nature of collective I/O, the access pattern of each individual process can be lost after I/O requests are aggregated at the parallel I/O middleware layer. In this study, we analyze this issue in detail. We show that such lost access pattern can have a negative impact on underlying caching algorithms' view of locality and can result in many unnecessary cache misses in low level buffer caches and additional disk accesses. To address this issue, we propose to reveal unseen access patterns - performing collective I/O but more importantly retaining applications' access patterns to underlying cache management. With such an idea, we have prototyped a new collective I/O aware cache management methodology. The evaluations with various cache management algorithms have confirmed clear advantages over the existing collective I/O strategy that throws away applications' original access pattern.

Original languageEnglish
Title of host publicationICS 2014 - Proceedings of the 28th ACM International Conference on Supercomputing
PublisherAssociation for Computing Machinery
Number of pages10
ISBN (Print)9781450326421
StatePublished - 2014
Event28th ACM International Conference on Supercomputing, ICS 2014 - Munich, Germany
Duration: Jun 10 2014Jun 13 2014

Publication series

NameProceedings of the International Conference on Supercomputing


Conference28th ACM International Conference on Supercomputing, ICS 2014


  • collective i/o
  • high performance computing
  • parallel i/o

Fingerprint Dive into the research topics of 'Revealing applications' access pattern in collective I/O for cache management'. Together they form a unique fingerprint.

Cite this