Toward Efficient Block Replication Management in Distributed Storage

Jianwei Liao, Zhibing Sha, Zhigang Cai, Zhiming Liu, Kenli Li, Wei Keng Liao, Alok N. Choudhary, Yutaka Ishiakwa

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Distributed/parallel file systems commonly suffer from load imbalance and resource contention due to the bursty characteristic exhibited in scientific applications. This article presents an adaptive scheme supporting dynamic block data replication and an efficient replica placement policy to improve the I/O performance of a distributed file system. Our goal is not only to yield a balanced data replication among storage servers but also a high degree of data access parallelism for the applications. We first present mathematical cost models to formulate the cost of data block replication by considering both the overhead and reduced data access time to the replicated data. To verify the validity and feasibility of the proposed cost model, we implement our proposal in a prototype distributed file system and evaluate it using a set of representative database-relevant application benchmarks. Our results demonstrate that the proposed approach can boost the usage efficiency of the data replicas with acceptable overhead of data replication management. Consequently, the overall data throughput of storage system can be noticeably improved. In summary, the proposed replication management scheme works well, especially for the database-relevant applications that exhibit an uneven access frequency and pattern to different parts of files.

Original languageEnglish (US)
Article number13
JournalACM Transactions on Modeling and Performance Evaluation of Computing Systems
Volume5
Issue number3
DOIs
StatePublished - Nov 2020

Funding

This work was partially supported by the National Natural Science Foundation of China (no. 61872299), Natural Science Foundation Project of CQ CSTC (no. CSTC2018jcyjAX0552), Hunan Provincial Natural Science Foundation of China (no. 2018JJ2309), and Fundamental Research Funds for the Central Universities (no. XDJK2017B044, XDJK2018D013). Authors’ addresses: J. Liao, Z. Sha, Z. Cai, and Z. Liu, College of Computer and Information Science, Southwest University of China, Beibei, Chongqing, China, 400715; emails: [email protected], [email protected], {czg, zhimingliu88}@swu.edu.cn; K. Li, College of Computer Science and Electronic Engineering, Hunan University, Chang-sha, Hunan, China, 410006; email: [email protected]; W. Liao and A. Choudhary, Department of Computer Science, North-western University, Chicago, USA, 60201; emails: [email protected], [email protected]; Y. Ishikawa, RIKEN Center for Computational Science, Chuo, Kobe, Hyogo, Japan, 650-0047; email: [email protected]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2020 Association for Computing Machinery. 2376-3639/2020/10-ART13 $15.00 https://doi.org/10.1145/3412450 This work was partially supported by the National Natural Science Foundation of China (no. 61872299), Natural Science Foundation Project of CQ CSTC (no. CSTC2018jcyjAX0552), Hunan Provincial Natural Science Foundation of China (no. 2018JJ2309), and Fundamental Research Funds for the Central Universities (no. XDJK2017B044, XDJK2018D013).

Keywords

  • Distributed file systems
  • access load balance
  • block data replication
  • modeling
  • replica placement

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Software
  • Information Systems
  • Media Technology
  • Safety, Risk, Reliability and Quality
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Toward Efficient Block Replication Management in Distributed Storage'. Together they form a unique fingerprint.

Cite this