TY - JOUR
T1 - A Holistic Heterogeneity-Aware Data Placement Scheme for Hybrid Parallel I/O Systems
AU - He, Shuibing
AU - Li, Zheng
AU - Zhou, Jiang
AU - Yin, Yanlong
AU - Xu, Xiaohua
AU - Chen, Yong
AU - Sun, Xian He
N1 - Funding Information:
This work was supported in part by the National Science Foundation of China No. 61572377, the Natural Science Foundation of Hubei Province of China No. 2017CFC889, the Fundamental Research Funds for the Central Universities No. 2018QNA5015, and the Zhejiang Lab Research Project No. 2019KC0AC01.
Publisher Copyright:
© 1990-2012 IEEE.
PY - 2020/4/1
Y1 - 2020/4/1
N2 - We present H2DP, a holistic heterogeneity-aware data placement scheme for hybrid parallel I/O systems, which consist of HDD servers and SSD servers. Most of the existing approaches focus on server performance or application I/O pattern heterogeneity in data placement. H2DP considers three axes of heterogeneity: server performance, server space, and application I/O pattern. More specifically, H2DP determines the optimized stripe sizes on servers based on server performance, keeps only critical data on all hybrid servers and the rest data on HDD servers, and dynamically migrates data among different types of servers at run-time. This holistic heterogeneity-awareness enables H2DP to achieve high performance by alleviating server load imbalance, efficiently utilizing SSD space, and accommodating application pattern variation. We have implemented a prototype of H2DP under MPICH2 atop OrangeFS. Extensive experimental results demonstrate that H2DP significantly improve I/O system performance compared to existing data placement schemes.
AB - We present H2DP, a holistic heterogeneity-aware data placement scheme for hybrid parallel I/O systems, which consist of HDD servers and SSD servers. Most of the existing approaches focus on server performance or application I/O pattern heterogeneity in data placement. H2DP considers three axes of heterogeneity: server performance, server space, and application I/O pattern. More specifically, H2DP determines the optimized stripe sizes on servers based on server performance, keeps only critical data on all hybrid servers and the rest data on HDD servers, and dynamically migrates data among different types of servers at run-time. This holistic heterogeneity-awareness enables H2DP to achieve high performance by alleviating server load imbalance, efficiently utilizing SSD space, and accommodating application pattern variation. We have implemented a prototype of H2DP under MPICH2 atop OrangeFS. Extensive experimental results demonstrate that H2DP significantly improve I/O system performance compared to existing data placement schemes.
KW - Parallel I/O system
KW - data placement
KW - hybrid parallel file system
KW - parallel file system
KW - solid state drive
UR - http://www.scopus.com/inward/record.url?scp=85078529312&partnerID=8YFLogxK
U2 - 10.1109/TPDS.2019.2948901
DO - 10.1109/TPDS.2019.2948901
M3 - Article
AN - SCOPUS:85078529312
VL - 31
SP - 830
EP - 842
JO - IEEE Transactions on Parallel and Distributed Systems
JF - IEEE Transactions on Parallel and Distributed Systems
SN - 1045-9219
IS - 4
M1 - 8880508
ER -