LoomIO: Object-Level Coordination in Distributed File Systems

Yusheng Hua, Xuanhua Shi, Kang He, Hai Jin, Wei Xie, Ligang He, Yong Chen

Research output: Contribution to journalArticlepeer-review

Abstract

Device-level interference is recognized as a major cause of the performance degradation in distributed file systems. Although the approaches of mitigating interference through coordination at application-level, middleware-level, and server-level have shown beneficial results in previous studies, we find their effectiveness is largely reduced since I/O requests are re-arranged by underlying object file systems. In this research study, we prove that object-level coordination is critical and often the key to address the interference issue, as the scheduling of object requests determines the device-level accesses and thus determines the actual I/O bandwidth and latency. This article proposes an object-level coordination system, LoomIO, which uses an OBOP (One-Broadcast-One-Propagate) method and a time-limited coordination process to deliver highly efficient coordination service. Specifically, LoomIO enables object requests to achieve an optimized scheduling decision within a few milliseconds and largely mitigates the device-level interference. We have implemented a LoomIO prototye and integrated it into Ceph file system. The evaluation results show that LoomIO achieved the considerable improvements in resource utilization (by up to 35%), in I/O throughput (by up to 31%), and in 99th percentile latency (by up to 54%) compared to the K-optimal method which uses the same scheduling algorithm as LoomIO but does not have the coordination support.

Original languageEnglish
Pages (from-to)1799-1810
Number of pages12
JournalIEEE Transactions on Parallel and Distributed Systems
Volume33
Issue number8
DOIs
StatePublished - Aug 1 2022

Keywords

  • Distributed object file system
  • Erasure-coding
  • I/O coordination
  • Performance

Fingerprint

Dive into the research topics of 'LoomIO: Object-Level Coordination in Distributed File Systems'. Together they form a unique fingerprint.

Cite this