Abstract
Distributed/parallel file systems commonly suffer from load imbalance and resource contention due to the bursty characteristic exhibited in scientific applications. This article presents an adaptive scheme supporting dynamic block data replication and an efficient replica placement policy to improve the I/O performance of a distributed file system. Our goal is not only to yield a balanced data replication among storage servers but also a high degree of data access parallelism for the applications. We first present mathematical cost models to formulate the cost of data block replication by considering both the overhead and reduced data access time to the replicated data. To verify the validity and feasibility of the proposed cost model, we implement our proposal in a prototype distributed file system and evaluate it using a set of representative database-relevant application benchmarks. Our results demonstrate that the proposed approach can boost the usage efficiency of the data replicas with acceptable overhead of data replication management. Consequently, the overall data throughput of storage system can be noticeably improved. In summary, the proposed replication management scheme works well, especially for the database-relevant applications that exhibit an uneven access frequency and pattern to different parts of files.
Original language | English (US) |
---|---|
Article number | 13 |
Journal | ACM Transactions on Modeling and Performance Evaluation of Computing Systems |
Volume | 5 |
Issue number | 3 |
DOIs | |
State | Published - Nov 2020 |
Funding
This work was partially supported by the National Natural Science Foundation of China (no. 61872299), Natural Science Foundation Project of CQ CSTC (no. CSTC2018jcyjAX0552), Hunan Provincial Natural Science Foundation of China (no. 2018JJ2309), and Fundamental Research Funds for the Central Universities (no. XDJK2017B044, XDJK2018D013). Authors’ addresses: J. Liao, Z. Sha, Z. Cai, and Z. Liu, College of Computer and Information Science, Southwest University of China, Beibei, Chongqing, China, 400715; emails: [email protected], [email protected], {czg, zhimingliu88}@swu.edu.cn; K. Li, College of Computer Science and Electronic Engineering, Hunan University, Chang-sha, Hunan, China, 410006; email: [email protected]; W. Liao and A. Choudhary, Department of Computer Science, North-western University, Chicago, USA, 60201; emails: [email protected], [email protected]; Y. Ishikawa, RIKEN Center for Computational Science, Chuo, Kobe, Hyogo, Japan, 650-0047; email: [email protected]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2020 Association for Computing Machinery. 2376-3639/2020/10-ART13 $15.00 https://doi.org/10.1145/3412450 This work was partially supported by the National Natural Science Foundation of China (no. 61872299), Natural Science Foundation Project of CQ CSTC (no. CSTC2018jcyjAX0552), Hunan Provincial Natural Science Foundation of China (no. 2018JJ2309), and Fundamental Research Funds for the Central Universities (no. XDJK2017B044, XDJK2018D013).
Keywords
- Distributed file systems
- access load balance
- block data replication
- modeling
- replica placement
ASJC Scopus subject areas
- Computer Science (miscellaneous)
- Software
- Information Systems
- Media Technology
- Safety, Risk, Reliability and Quality
- Hardware and Architecture
- Computer Networks and Communications