Toward Efficient Block Replication Management in Distributed Storage

Jianwei Liao, Zhibing Sha, Zhigang Cai, Zhiming Liu, Kenli Li, Wei Keng Liao, Alok N. Choudhary, Yutaka Ishiakwa

Research output: Contribution to journalArticlepeer-review

Abstract

Distributed/parallel file systems commonly suffer from load imbalance and resource contention due to the bursty characteristic exhibited in scientific applications. This article presents an adaptive scheme supporting dynamic block data replication and an efficient replica placement policy to improve the I/O performance of a distributed file system. Our goal is not only to yield a balanced data replication among storage servers but also a high degree of data access parallelism for the applications. We first present mathematical cost models to formulate the cost of data block replication by considering both the overhead and reduced data access time to the replicated data. To verify the validity and feasibility of the proposed cost model, we implement our proposal in a prototype distributed file system and evaluate it using a set of representative database-relevant application benchmarks. Our results demonstrate that the proposed approach can boost the usage efficiency of the data replicas with acceptable overhead of data replication management. Consequently, the overall data throughput of storage system can be noticeably improved. In summary, the proposed replication management scheme works well, especially for the database-relevant applications that exhibit an uneven access frequency and pattern to different parts of files.

Original languageEnglish (US)
Article number13
JournalACM Transactions on Modeling and Performance Evaluation of Computing Systems
Volume5
Issue number3
DOIs
StatePublished - Nov 2020

Keywords

  • Distributed file systems
  • access load balance
  • block data replication
  • modeling
  • replica placement

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems
  • Software
  • Safety, Risk, Reliability and Quality
  • Media Technology

Fingerprint

Dive into the research topics of 'Toward Efficient Block Replication Management in Distributed Storage'. Together they form a unique fingerprint.

Cite this