Dynamic active storage for high performance I/O

Many high-end computing applications in critical areas of science and technology are becoming more and more data intensive. These applications transfer large amounts of data from storage nodes to compute nodes for processing, which is costly and bandwidth consuming. The data movement often dominates the applications' run time. Active storage provides a promising solution for these applications by moving appropriate computations from compute nodes to storage nodes. The prior research has achieved considerable progress and developed several active storage models. However, the existing studies have neglected the influence of data dependence on the performance of active storage systems. This study shows that the data dependence has a critical impact on active storage, and the ignorance of dependence can lead to waste of the precious bandwidth. To address this issue in active storage, this paper also presents a new Dynamic Active Storage (DAS) architecture that analyzes the bandwidth requirement of operations, determines the applicability for active storage requests, and optimizes data layout on servers to minimize the bandwidth requirement. Experimental tests have been conducted, and the results have confirmed that the proposed DAS architecture outperforms existing active storage systems. The DAS architecture reduces the data movement caused by data dependence and improves applications' performance over existing schemes. It holds a promise for high performance I/O system in high-end computing.