Hashing Based Data Distribution in Heterogeneous Storage

Jiang Zhou, Lin Su, Weiping Wang, Yong Chen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Storage systems are important infrastructures of cloud computing in data centers. They demand the efficiency to distribute data and provide high I/O performance. The consistent hashing algorithm is widely used in modern storage systems due to its decentralized design, scalability, and adaptability to node changes. However, it lacks efficiency in a heterogeneous environment. In this study, we propose a Dynamically Attributed Consistent Hashing (DACH), to overcome this deficiency. DACH manages heterogeneous storage resources on consistent hashing ring and maintains multiple, dynamic attributes for nodes to characterize their distinct features. It places data on the ring and selects nodes with a balanced, weighted data distribution algorithm by taking full advantage of node attributes. By considering attribute variation, such as remaining capacity and workload changes, DACH can further optimize data layout. Extensive evaluation results show that, by well exploiting storage heterogeneity, DACH achieves adaptive and efficient data distribution for heterogeneous storage systems.

Original languageEnglish
Title of host publication19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages652-659
Number of pages8
ISBN (Electronic)9781665435741
DOIs
StatePublished - 2021
Event19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021 - New York, United States
Duration: Sep 30 2021Oct 3 2021

Publication series

Name19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021

Conference

Conference19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021
Country/TerritoryUnited States
CityNew York
Period09/30/2110/3/21

Keywords

  • Consistent hashing
  • Data distribution
  • Heterogeneous storage
  • Non-volatile memory

Fingerprint

Dive into the research topics of 'Hashing Based Data Distribution in Heterogeneous Storage'. Together they form a unique fingerprint.

Cite this