TY - GEN
T1 - Revealing applications' access pattern in collective I/O for cache management
AU - Lu, Yin
AU - Chen, Yong
AU - Latham, Rob
AU - Zhuang, Yu
PY - 2014
Y1 - 2014
N2 - Collective I/O is a critical I/O strategy on high-performance parallel computing systems that enables programmers to reveal parallel processes' I/O accesses collectively and makes possible for the parallel I/O middleware to carry out I/O requests in a highly efficient manner. Collective I/O has been proven as a core parallel I/O optimization technique. However, due to the collective nature of collective I/O, the access pattern of each individual process can be lost after I/O requests are aggregated at the parallel I/O middleware layer. In this study, we analyze this issue in detail. We show that such lost access pattern can have a negative impact on underlying caching algorithms' view of locality and can result in many unnecessary cache misses in low level buffer caches and additional disk accesses. To address this issue, we propose to reveal unseen access patterns - performing collective I/O but more importantly retaining applications' access patterns to underlying cache management. With such an idea, we have prototyped a new collective I/O aware cache management methodology. The evaluations with various cache management algorithms have confirmed clear advantages over the existing collective I/O strategy that throws away applications' original access pattern.
AB - Collective I/O is a critical I/O strategy on high-performance parallel computing systems that enables programmers to reveal parallel processes' I/O accesses collectively and makes possible for the parallel I/O middleware to carry out I/O requests in a highly efficient manner. Collective I/O has been proven as a core parallel I/O optimization technique. However, due to the collective nature of collective I/O, the access pattern of each individual process can be lost after I/O requests are aggregated at the parallel I/O middleware layer. In this study, we analyze this issue in detail. We show that such lost access pattern can have a negative impact on underlying caching algorithms' view of locality and can result in many unnecessary cache misses in low level buffer caches and additional disk accesses. To address this issue, we propose to reveal unseen access patterns - performing collective I/O but more importantly retaining applications' access patterns to underlying cache management. With such an idea, we have prototyped a new collective I/O aware cache management methodology. The evaluations with various cache management algorithms have confirmed clear advantages over the existing collective I/O strategy that throws away applications' original access pattern.
KW - collective i/o
KW - high performance computing
KW - parallel i/o
UR - http://www.scopus.com/inward/record.url?scp=84903752716&partnerID=8YFLogxK
U2 - 10.1145/2597652.2597686
DO - 10.1145/2597652.2597686
M3 - Conference contribution
AN - SCOPUS:84903752716
SN - 9781450326421
T3 - Proceedings of the International Conference on Supercomputing
SP - 181
EP - 190
BT - ICS 2014 - Proceedings of the 28th ACM International Conference on Supercomputing
PB - Association for Computing Machinery
T2 - 28th ACM International Conference on Supercomputing, ICS 2014
Y2 - 10 June 2014 through 13 June 2014
ER -