Hierarchical consistent hashing for heterogeneous object-based storage

Jiang Zhou, Wei Xie, Qiang Gu, Yong Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Distributed storage systems play an increasingly critical role in data centers to meet the ever-increasing data growth demand. Heterogeneous storage systems, with the coexistence of hard disk drives (HDDs) and solid state drives (SSDs), can be an attractive distributed store solution due to the balanced performance, large capacity, and economic cost. The consistent hashing distribution algorithm that is widely used in distributed storage systems can achieve scalable and flexible mappings, but do not take full advantages of different characteristics of heterogeneous devices. In this research, we propose a hierarchical consistent hashing (HiCH) algorithm to better manage data distribution in a heterogeneous object-based storage system and better explore the potential of heterogeneous devices. HiCH divides heterogeneous storage devices into different buckets and applies separate consistent hashing rings for each bucket. It places data into various hashing rings according to the hotness, access time, and other data access patterns. The evaluation based on the Sheepdog, a distributed objectbased storage system, confirms that HiCH can improve the performance of storage systems, and also make better utilization of heterogeneous storage devices.

Original languageEnglish
Title of host publicationProceedings - 15th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, 10th IEEE International Conference on Big Data Science and Engineering and 14th IEEE International Symposium on Parallel and Distributed Processing with Applications, IEEE TrustCom/BigDataSE/ISPA 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1597-1604
Number of pages8
ISBN (Electronic)9781509032051
DOIs
StatePublished - 2016
EventJoint 15th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, 10th IEEE International Conference on Big Data Science and Engineering and 14th IEEE International Symposium on Parallel and Distributed Processing with Applications, IEEE TrustCom/BigDataSE/ISPA 2016 - Tianjin, China
Duration: Aug 23 2016Aug 26 2016

Publication series

NameProceedings - 15th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, 10th IEEE International Conference on Big Data Science and Engineering and 14th IEEE International Symposium on Parallel and Distributed Processing with Applications, IEEE TrustCom/BigDataSE/ISPA 2016

Conference

ConferenceJoint 15th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, 10th IEEE International Conference on Big Data Science and Engineering and 14th IEEE International Symposium on Parallel and Distributed Processing with Applications, IEEE TrustCom/BigDataSE/ISPA 2016
CountryChina
CityTianjin
Period08/23/1608/26/16

Fingerprint Dive into the research topics of 'Hierarchical consistent hashing for heterogeneous object-based storage'. Together they form a unique fingerprint.

  • Cite this

    Zhou, J., Xie, W., Gu, Q., & Chen, Y. (2016). Hierarchical consistent hashing for heterogeneous object-based storage. In Proceedings - 15th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, 10th IEEE International Conference on Big Data Science and Engineering and 14th IEEE International Symposium on Parallel and Distributed Processing with Applications, IEEE TrustCom/BigDataSE/ISPA 2016 (pp. 1597-1604). [7847130] (Proceedings - 15th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, 10th IEEE International Conference on Big Data Science and Engineering and 14th IEEE International Symposium on Parallel and Distributed Processing with Applications, IEEE TrustCom/BigDataSE/ISPA 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/TrustCom.2016.0247