A cost-intelligent application-specific data layout scheme for parallel file systems

Huaiming Song, Yanlong Yin, Yong Chen, Xian-He Sun

Research output: Contribution to conferencePaper

Abstract

I/O data access is a recognized performance bottleneck of high-end computing. Several commercial and research parallel file systems have been developed in recent years to ease the performance bottleneck. These advanced file systems perform well on some applications but may not perform well on others. They have not reached their full potential in mitigating the I/O-wall problem. Data access is application dependent. Based on the application-specific optimization principle, in this study we propose a cost-intelligent data access strategy to improve the performance of parallel file systems. We first present a novel model to estimate data access cost of different data layout policies. Next, we extend the cost model to calculate the overall I/O cost of any given application and choose an appropriate layout policy for the application. A complex application may consist of different data access patterns. Averaging the data access patterns may not be the best solution for those complex applications
Original languageEnglish
StatePublished - Jun 8 2011

Fingerprint Dive into the research topics of 'A cost-intelligent application-specific data layout scheme for parallel file systems'. Together they form a unique fingerprint.

  • Cite this