Using Working Set Reorganization to Manage Storage Systems with Hard and Solid State Disks

Junjie Chen, Jialin Liu, Philip Roth, Yong Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Scientific applications from many problem domains produce and/or access large volumes of data. To support these applications, designers of high-end computing (HEC) systems have greatly increased the capacity of storage systems in recent years. However, because hard disk drives (HDDs) are still the dominant storage device used in HEC storage systems, and because HDD performance has not improved as quickly as the capacity, it can be challenging to deploy a storage system that provides both extreme capacity and extreme performance at a reasonable cost. Solid State Drives (SSDs) are a promising high-bandwidth and low-latency alternative to HDDs for HEC storage systems, but they too have deficiencies: small capacity, limited write cycles, and high cost when compared to HDDs. Because of their complementary characteristics, storage system designers are beginning to consider heterogeneous storage system designs that include both HDDs and SSDs. However, managing the workload so as to take advantage of the strengths of each type of storage device while controlling overhead is a major challenge. In this study, we propose a novel approach for managing a heterogeneous storage system called the Working Set-based Reorganization Scheme (WS-ROS). With WS-ROS, applications write to both HDDs and SSDs using all the available storage system bandwidth. Later, a background process reorganizes the data so as to place the data most likely to be read on SSDs while relegating the data most likely to be written and the data not likely to be accessed onto the slower but higher-capacity HDDs. For our evaluation workloads, the WS-ROS approach provided a 3× to 10× performance improvement compared to a heterogeneous storage system without a working set-based data reorganization scheme, suggesting the value of lazy reorganization of data based on data access working sets.

Original languageEnglish
Title of host publicationProceedings - 43rd International Conference on Parallel Processing Workshops, ICPPW 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages283-291
Number of pages9
ISBN (Electronic)9781479956159
DOIs
StatePublished - May 7 2015
Event43rd International Conference on Parallel Processing Workshops, ICPPW 2014 - Minneapolis, United States
Duration: Sep 9 2014Sep 12 2014

Publication series

NameProceedings of the International Conference on Parallel Processing Workshops
Volume2015-May
ISSN (Print)1530-2016

Conference

Conference43rd International Conference on Parallel Processing Workshops, ICPPW 2014
CountryUnited States
CityMinneapolis
Period09/9/1409/12/14

Keywords

  • High-end computing
  • Solid State Drives
  • data-intensive computing
  • parallel file systems
  • storage

Fingerprint Dive into the research topics of 'Using Working Set Reorganization to Manage Storage Systems with Hard and Solid State Disks'. Together they form a unique fingerprint.

  • Cite this

    Chen, J., Liu, J., Roth, P., & Chen, Y. (2015). Using Working Set Reorganization to Manage Storage Systems with Hard and Solid State Disks. In Proceedings - 43rd International Conference on Parallel Processing Workshops, ICPPW 2014 (pp. 283-291). [7103463] (Proceedings of the International Conference on Parallel Processing Workshops; Vol. 2015-May). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICPPW.2014.45