Dynamic active storage for high performance I/O

Chao Chen, Yong Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Many high-end computing applications in critical areas of science and technology are becoming more and more data intensive. These applications transfer large amounts of data from storage nodes to compute nodes for processing, which is costly and bandwidth consuming. The data movement often dominates the applications' run time. Active storage provides a promising solution for these applications by moving appropriate computations from compute nodes to storage nodes. The prior research has achieved considerable progress and developed several active storage models. However, the existing studies have neglected the influence of data dependence on the performance of active storage systems. This study shows that the data dependence has a critical impact on active storage, and the ignorance of dependence can lead to waste of the precious bandwidth. To address this issue in active storage, this paper also presents a new Dynamic Active Storage (DAS) architecture that analyzes the bandwidth requirement of operations, determines the applicability for active storage requests, and optimizes data layout on servers to minimize the bandwidth requirement. Experimental tests have been conducted, and the results have confirmed that the proposed DAS architecture outperforms existing active storage systems. The DAS architecture reduces the data movement caused by data dependence and improves applications' performance over existing schemes. It holds a promise for high performance I/O system in high-end computing.

Original languageEnglish
Title of host publicationProceedings - 41st International Conference on Parallel Processing, ICPP 2012
Pages379-388
Number of pages10
DOIs
StatePublished - Dec 20 2012
Event41st International Conference on Parallel Processing, ICPP 2012 - Pittsburgh, PA, United States
Duration: Sep 10 2012Sep 13 2012

Publication series

NameProceedings of the International Conference on Parallel Processing
ISSN (Print)0190-3918

Conference

Conference41st International Conference on Parallel Processing, ICPP 2012
CountryUnited States
CityPittsburgh, PA
Period09/10/1209/13/12

    Fingerprint

Keywords

  • active storage
  • data intensive computing
  • dynamic active storage
  • high end computing
  • parallel I/O
  • parallel file systems

Cite this

Chen, C., & Chen, Y. (2012). Dynamic active storage for high performance I/O. In Proceedings - 41st International Conference on Parallel Processing, ICPP 2012 (pp. 379-388). [6337599] (Proceedings of the International Conference on Parallel Processing). https://doi.org/10.1109/ICPP.2012.22