Attributed consistent hashing for heterogeneous storage systems

Jiang Zhou, Yong Chen, Weiping Wang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Storage systems are critical building blocks of high-end computing systems and data centers. They demand the flexibility to distribute data effectively and provide high I/O performance. The consistent hashing algorithm is widely used in parallel/distributed file systems due to its decentralized design, scalability, and adaptability to node changes. However, it lacks efficiency in a heterogeneous environment where different storage devices, e.g. hard disk drives and solid state drives, co-exist. In this study, we propose an attributed consistent hashing (attributedCH), to overcome this deficiency. AttributedCH manages heterogeneous nodes on a consistent hashing ring and maintains attributes for each node to characterize distinct node features. It divides the hash ring into sectors and selects nodes from the sector with a comprehensive selection strategy. By considering different attributes, attributedCH achieves adaptive and efficient data placement for heterogeneous storage systems. We have carried out extensive evaluations and the evaluation results confirm that the attributedCH overcomes the deficiency of existing consistent hashing algorithms well and is particularly suitable for heterogeneous storage systems.

Original languageEnglish
Title of host publicationProceedings - 27th International Conference on Parallel Architectures and Compilation Techniques, PACT 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781450359863
DOIs
StatePublished - Nov 1 2018
Event27th International Conference on Parallel Architectures and Compilation Techniques, PACT 2018 - Limassol, Cyprus
Duration: Nov 1 2018Nov 4 2018

Publication series

NameParallel Architectures and Compilation Techniques - Conference Proceedings, PACT
ISSN (Print)1089-795X

Conference

Conference27th International Conference on Parallel Architectures and Compilation Techniques, PACT 2018
CountryCyprus
CityLimassol
Period11/1/1811/4/18

Keywords

  • Consistent hashing
  • Data distribution
  • Heterogeneous storage system
  • Parallel file system

Fingerprint Dive into the research topics of 'Attributed consistent hashing for heterogeneous storage systems'. Together they form a unique fingerprint.

Cite this