TY - GEN
T1 - ECCH
T2 - 19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021
AU - Xiong, Yinglong
AU - Zhou, Jiang
AU - Su, Lin
AU - Wang, Weiping
AU - Chen, Yong
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - In this paper, we propose ECCH, an Erasure Coded Consistent Hashing scheme to make better data placement in distributed storage systems. It combines the inherent advantages of consistent hashing together with the storage-efficiency of erasure coding technology. Specifically, ECCH divides data block stream of files into groups according to block IDs. In each group, it encoded data blocks with additional parity blocks by erasure coding. All encoded blocks in the same group are stored on different nodes with consistent hashing distribution. For node failure or data loss, ECCH locates required data through the ID of missing block in a same group for fast recovery. To deal with node changes, ECCH introduces a design of multi-version hash rings to manage data layout. It can prevent the impact of data migration on erasure coding, while achieving data balance with little data movement. We have implemented ECCH on the Sheepdog, a distributed object-based storage system. Evaluation results show that ECCH can greatly improve the space utilization of hashing-based storage systems, while achieving efficient fault tolerance.
AB - In this paper, we propose ECCH, an Erasure Coded Consistent Hashing scheme to make better data placement in distributed storage systems. It combines the inherent advantages of consistent hashing together with the storage-efficiency of erasure coding technology. Specifically, ECCH divides data block stream of files into groups according to block IDs. In each group, it encoded data blocks with additional parity blocks by erasure coding. All encoded blocks in the same group are stored on different nodes with consistent hashing distribution. For node failure or data loss, ECCH locates required data through the ID of missing block in a same group for fast recovery. To deal with node changes, ECCH introduces a design of multi-version hash rings to manage data layout. It can prevent the impact of data migration on erasure coding, while achieving data balance with little data movement. We have implemented ECCH on the Sheepdog, a distributed object-based storage system. Evaluation results show that ECCH can greatly improve the space utilization of hashing-based storage systems, while achieving efficient fault tolerance.
KW - Consistent hashing
KW - Data distribution
KW - Erasure coding
KW - Fault tolerance
KW - Replication
UR - http://www.scopus.com/inward/record.url?scp=85124120385&partnerID=8YFLogxK
U2 - 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00036
DO - 10.1109/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00036
M3 - Conference contribution
AN - SCOPUS:85124120385
T3 - 19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021
SP - 177
EP - 184
BT - 19th IEEE International Symposium on Parallel and Distributed Processing with Applications, 11th IEEE International Conference on Big Data and Cloud Computing, 14th IEEE International Conference on Social Computing and Networking and 11th IEEE International Conference on Sustainable Computing and Communications, ISPA/BDCloud/SocialCom/SustainCom 2021
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 30 September 2021 through 3 October 2021
ER -