TY - GEN
T1 - A new data sieving approach for high performance I/O
AU - Lu, Yin
AU - Chen, Yong
AU - Amritkar, Prathamesh
AU - Thakur, Rajeev
AU - Zhuang, Yu
PY - 2012
Y1 - 2012
N2 - Many scientific computing applications and engineering simulations exhibit noncontiguous I/O access patterns. Data sieving is an important technique to improve the performance of noncontiguous I/O accesses by combining small and noncontiguous requests into a large and contiguous request. It has been proven effective even though more data is potentially accessed than demanded. In this study, we propose a new data sieving approach namely Performance Model Directed Data Sieving, or PMD data sieving in short. It improves the existing data sieving approach from two aspects: (1) dynamically determines when it is beneficial to perform data sieving; and (2) dynamically determines how to perform data sieving if beneficial. It improves the performance of the existing data sieving approach and reduces the memory consumption as verified by experimental results. Given the importance of supporting noncontiguous accesses effectively and reducing the memory pressure in a large-scale system, the proposed PMD data sieving approach in this research holds a promise and will have an impact on high performance I/O systems.
AB - Many scientific computing applications and engineering simulations exhibit noncontiguous I/O access patterns. Data sieving is an important technique to improve the performance of noncontiguous I/O accesses by combining small and noncontiguous requests into a large and contiguous request. It has been proven effective even though more data is potentially accessed than demanded. In this study, we propose a new data sieving approach namely Performance Model Directed Data Sieving, or PMD data sieving in short. It improves the existing data sieving approach from two aspects: (1) dynamically determines when it is beneficial to perform data sieving; and (2) dynamically determines how to perform data sieving if beneficial. It improves the performance of the existing data sieving approach and reduces the memory consumption as verified by experimental results. Given the importance of supporting noncontiguous accesses effectively and reducing the memory pressure in a large-scale system, the proposed PMD data sieving approach in this research holds a promise and will have an impact on high performance I/O systems.
KW - Data sieving
KW - High performance computing
KW - Libraries
KW - Parallel I/O
KW - Parallel file systems
KW - Runtime systems
UR - http://www.scopus.com/inward/record.url?scp=84867049136&partnerID=8YFLogxK
U2 - 10.1007/978-94-007-4516-2_12
DO - 10.1007/978-94-007-4516-2_12
M3 - Conference contribution
AN - SCOPUS:84867049136
SN - 9789400745155
T3 - Lecture Notes in Electrical Engineering
SP - 111
EP - 121
BT - Future Information Technology, Application, and Service, FutureTech 2012
Y2 - 26 June 2012 through 28 June 2012
ER -