TY - GEN
T1 - RE-Store
T2 - 2019 IEEE International Conference on Cluster Computing, CLUSTER 2019
AU - Li, Yuzhe
AU - Zhou, Jiang
AU - Wang, Weiping
AU - Chen, Yong
N1 - Publisher Copyright:
© 2019 IEEE.
PY - 2019/9
Y1 - 2019/9
N2 - In-memory key/value stores (KV-stores) are a key building block for numerous applications running on a cluster. As cluster scales have grown, efficiency and availability have become increasingly critical characteristics. Traditional replication provides redundancy, but is inefficient due to its high storage overhead. Erasure coding can provide data reliability with significantly lower storage requirements, but is primarily used for long-term archival data due to the limitation of its write performance. Recent studies have attempted to combine these two techniques by using replication for frequently-updated metadata, and erasure coding for large, read-only data. In this study, we propose RE-Store, an in-memory key/value store system which utilizes a novel hybrid replication/erasure coding scheme to achieve both efficiency and reliability. RE-Store introduces replication into erasure coding by making one copy of each encoded datum and replacing partial parity with replicas for improved storage-efficiency. When failures occur, it uses these replicas to ensure data availability and thus avoids the inefficiencies of erasure coding during repair. RE-Store provides fault tolerance through fast, online recovery during different failure scenarios with little performance degradation. We have implemented RE-Store on a real key/value system and conducted extensive evaluations to validate its design and to study its performance, efficiency, and reliability. Experimental results show that RE-Store performs similarly to erasure coding and replication under normal operations while saving 18% to 34% of the memory used by replication when tolerating 2 to 4 failures.
AB - In-memory key/value stores (KV-stores) are a key building block for numerous applications running on a cluster. As cluster scales have grown, efficiency and availability have become increasingly critical characteristics. Traditional replication provides redundancy, but is inefficient due to its high storage overhead. Erasure coding can provide data reliability with significantly lower storage requirements, but is primarily used for long-term archival data due to the limitation of its write performance. Recent studies have attempted to combine these two techniques by using replication for frequently-updated metadata, and erasure coding for large, read-only data. In this study, we propose RE-Store, an in-memory key/value store system which utilizes a novel hybrid replication/erasure coding scheme to achieve both efficiency and reliability. RE-Store introduces replication into erasure coding by making one copy of each encoded datum and replacing partial parity with replicas for improved storage-efficiency. When failures occur, it uses these replicas to ensure data availability and thus avoids the inefficiencies of erasure coding during repair. RE-Store provides fault tolerance through fast, online recovery during different failure scenarios with little performance degradation. We have implemented RE-Store on a real key/value system and conducted extensive evaluations to validate its design and to study its performance, efficiency, and reliability. Experimental results show that RE-Store performs similarly to erasure coding and replication under normal operations while saving 18% to 34% of the memory used by replication when tolerating 2 to 4 failures.
KW - KV-store
KW - availability and reliability
KW - erasure coding
KW - fault tolerance
KW - replication
UR - http://www.scopus.com/inward/record.url?scp=85075268626&partnerID=8YFLogxK
U2 - 10.1109/CLUSTER.2019.8891013
DO - 10.1109/CLUSTER.2019.8891013
M3 - Conference contribution
AN - SCOPUS:85075268626
T3 - Proceedings - IEEE International Conference on Cluster Computing, ICCC
BT - Proceedings - 2019 IEEE International Conference on Cluster Computing, CLUSTER 2019
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 23 September 2019 through 26 September 2019
ER -