I/O analysis and optimization for an AMR cosmology application

Jianwei Li, Wei Keng Liao, A. Choudhary, V. Taylor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Scopus citations

Abstract

In this paper we investigate the data access patterns and file I/O behaviors of a production cosmology application that uses the adaptive mesh refinement (AMR) technique for its domain decomposition. This application was originally developed using Hierarchical Data Format (HDF version 4) I/O library and since HDF4 does not provide parallel I/O facilities, the global file I/O operations were carried out by one of the allocated processors. When the number of processors becomes large, the I/O performance of this design degrades significantly due to the high communication cost and sequential file access. In this work, we present two additional I/O implementations, using MPI-IO and parallel HDF version 5, and analyze their impacts to the I/O performance for this typical AMR application. Based on the I/O patterns discovered in this application, we also discuss the interaction between user level parallel I/O operations and different parallel file systems and point out the advantages and disadvantages. The performance results presented in this work are obtained from an SGI Origin2000 using XFS, an IBM SP using GPFS, and a Linux cluster using PVFS.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE International Conference on Cluster Computing, CLUSTER 2002
EditorsBill Gropp, Rajkumar Buyya, Rob Pennington, Maxine Brown, Mark Baker, Dan Reed
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages119-126
Number of pages8
ISBN (Electronic)0769517455
DOIs
StatePublished - Jan 1 2002
EventIEEE International Conference on Cluster Computing, CLUSTER 2002 - Chicago, United States
Duration: Sep 23 2002Sep 26 2002

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
Volume2002-January
ISSN (Print)1552-5244

Other

OtherIEEE International Conference on Cluster Computing, CLUSTER 2002
CountryUnited States
CityChicago
Period9/23/029/26/02

Keywords

  • Adaptive mesh refinement
  • Computational modeling
  • Concurrent computing
  • Costs
  • Degradation
  • File systems
  • Libraries
  • Linux
  • Parallel processing
  • Production

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Signal Processing

Fingerprint Dive into the research topics of 'I/O analysis and optimization for an AMR cosmology application'. Together they form a unique fingerprint.

Cite this