TY - GEN
T1 - Hashing Based Data Distribution in Heterogeneous Storage
AU - Zhou, Jiang
AU - Su, Lin
AU - Wang, Weiping
AU - Chen, Yong
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Storage systems are important infrastructures of cloud computing in data centers. They demand the efficiency to distribute data and provide high I/O performance. The consistent hashing algorithm is widely used in modern storage systems due to its decentralized design, scalability, and adaptability to node changes. However, it lacks efficiency in a heterogeneous environment. In this study, we propose a Dynamically Attributed Consistent Hashing (DACH), to overcome this deficiency. DACH manages heterogeneous storage resources on consistent hashing ring and maintains multiple, dynamic attributes for nodes to characterize their distinct features. It places data on the ring and selects nodes with a balanced, weighted data distribution algorithm by taking full advantage of node attributes. By considering attribute variation, such as remaining capacity and workload changes, DACH can further optimize data layout. Extensive evaluation results show that, by well exploiting storage heterogeneity, DACH achieves adaptive and efficient data distribution for heterogeneous storage systems.
AB - Storage systems are important infrastructures of cloud computing in data centers. They demand the efficiency to distribute data and provide high I/O performance. The consistent hashing algorithm is widely used in modern storage systems due to its decentralized design, scalability, and adaptability to node changes. However, it lacks efficiency in a heterogeneous environment. In this study, we propose a Dynamically Attributed Consistent Hashing (DACH), to overcome this deficiency. DACH manages heterogeneous storage resources on consistent hashing ring and maintains multiple, dynamic attributes for nodes to characterize their distinct features. It places data on the ring and selects nodes with a balanced, weighted data distribution algorithm by taking full advantage of node attributes. By considering attribute variation, such as remaining capacity and workload changes, DACH can further optimize data layout. Extensive evaluation results show that, by well exploiting storage heterogeneity, DACH achieves adaptive and efficient data distribution for heterogeneous storage systems.
KW - Consistent hashing
KW - Data distribution
KW - Heterogeneous storage
KW - Non-volatile memory
UR - http://www.scopus.com/inward/record.url?scp=85124134882&partnerID=8YFLogxK
U2 - 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00095
DO - 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00095
M3 - Conference contribution
AN - SCOPUS:85124134882
T3 - 19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021
SP - 652
EP - 659
BT - 19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021
Y2 - 30 September 2021 through 3 October 2021
ER -