Noncontiguous I/O through PVFS

Avery Ching, A. Choudhary, Wei Keng Liao, R. Ross, W. Gropp

Research output: Chapter in Book/Report/Conference proceedingConference contribution

46 Scopus citations

Abstract

With the tremendous advances in processor and memory technology, I/O has risen to become the bottleneck in high-performance computing for many applications. The development of parallel file systems has helped to ease the performance gap, but I/O still remains an area needing significant performance improvement. Research has found that noncontiguous I/O access patterns in scientific applications combined with current file system methods, to perform these accesses lead to unacceptable performance for large data sets. To enhance performance of noncontiguous I/O, we have created list I/O, a native version of noncontiguous I/O. We have used the Parallel Virtual File System (PVFS) to implement our ideas. Our research and experimentation shows that list I/O outperforms current noncontiguous I/O access methods in most I/O situations and can substantially enhance the performance of real-world scientific applications.

Original languageEnglish (US)
Title of host publicationProceedings - IEEE International Conference on Cluster Computing, CLUSTER 2002
EditorsBill Gropp, Rajkumar Buyya, Rob Pennington, Maxine Brown, Mark Baker, Dan Reed
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages405-414
Number of pages10
ISBN (Electronic)0769517455
DOIs
StatePublished - 2002
EventIEEE International Conference on Cluster Computing, CLUSTER 2002 - Chicago, United States
Duration: Sep 23 2002Sep 26 2002

Publication series

NameProceedings - IEEE International Conference on Cluster Computing, ICCC
Volume2002-January
ISSN (Print)1552-5244

Other

OtherIEEE International Conference on Cluster Computing, CLUSTER 2002
Country/TerritoryUnited States
CityChicago
Period9/23/029/26/02

Funding

of a large number of farmers to allow access to their land. Initial setup and maintenance of the study catchments was funded by research grants (DP0209724, DP0343778 and DP0556941) from the Australian Research Council, the CRC for Catchment Hydrology and NASA. The National Airborne Field Experiments have been made possible through recent infrastructure (LE0453434 and LE0560930) and research (DP0557543 and DP0556941) funding from the Australian Research Council, and the willingness

Keywords

  • Application software
  • Computer science
  • Concurrent computing
  • Costs
  • Distributed computing
  • File servers
  • File systems
  • Linux
  • Mathematics
  • Network servers

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Signal Processing

Fingerprint

Dive into the research topics of 'Noncontiguous I/O through PVFS'. Together they form a unique fingerprint.

Cite this