A new data sieving approach for high performance I/O

Yin Lu, Yong Chen, Prathamesh Amritkar, Rajeev Thakur, Yu Zhuang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Many scientific computing applications and engineering simulations exhibit noncontiguous I/O access patterns. Data sieving is an important technique to improve the performance of noncontiguous I/O accesses by combining small and noncontiguous requests into a large and contiguous request. It has been proven effective even though more data is potentially accessed than demanded. In this study, we propose a new data sieving approach namely Performance Model Directed Data Sieving, or PMD data sieving in short. It improves the existing data sieving approach from two aspects: (1) dynamically determines when it is beneficial to perform data sieving; and (2) dynamically determines how to perform data sieving if beneficial. It improves the performance of the existing data sieving approach and reduces the memory consumption as verified by experimental results. Given the importance of supporting noncontiguous accesses effectively and reducing the memory pressure in a large-scale system, the proposed PMD data sieving approach in this research holds a promise and will have an impact on high performance I/O systems.

Original languageEnglish
Title of host publicationFuture Information Technology, Application, and Service, FutureTech 2012
Pages111-121
Number of pages11
EditionVOL. 1
DOIs
StatePublished - 2012
Event7th FTRA International Conference on Future Information Technology, FutureTech 2012 - Vancouver, BC, Canada
Duration: Jun 26 2012Jun 28 2012

Publication series

NameLecture Notes in Electrical Engineering
NumberVOL. 1
Volume164 LNEE
ISSN (Print)1876-1100
ISSN (Electronic)1876-1119

Conference

Conference7th FTRA International Conference on Future Information Technology, FutureTech 2012
CountryCanada
CityVancouver, BC
Period06/26/1206/28/12

Keywords

  • Data sieving
  • High performance computing
  • Libraries
  • Parallel I/O
  • Parallel file systems
  • Runtime systems

Fingerprint Dive into the research topics of 'A new data sieving approach for high performance I/O'. Together they form a unique fingerprint.

Cite this