Design and implementation of a parallel I/O runtime system for irregular applications

Jaechun No*, Sung Soon Park, Jesús Carretero, Alok Nidhi Choudhary, Pang Chen

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper we present the design, implementation and evaluation of a runtime system based on collective I/O techniques for irregular applications. We present two models, namely, «Collective I/O» and «Pipelined Collective I/O». In the first scheme, all processors participate in the I/O simultaneously making scheduling of I/O requests simpler but creating a possibility of contention at the I/O nodes. In the second approach, processors are grouped into several groups, so that only one group performs I/O simultaneously, while the next group performs communication to rearrange data, and this entire process is pipelined to reduce I/O node contention dynamically. Both models have been optimized by using software caching, chunking and on-line compression mechanisms. We demonstrate that we can obtain significantly high-performance for I/O above what has been possible so far. The performance results are presented on an Intel Paragon and on the ASCI/Red teraflops machine at Sandia National Labs.

Original languageEnglish (US)
Title of host publicationProceedings of the 1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, IPPS/SPDP 1998
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages280-284
Number of pages5
ISBN (Electronic)0818684038, 9780818684036
DOIs
StatePublished - Jan 1 1998
Event1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, IPPS/SPDP 1998 - Orlando, United States
Duration: Mar 30 1998Apr 3 1998

Publication series

NameProceedings of the 1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, IPPS/SPDP 1998
Volume1998-March

Other

Other1st Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing, IPPS/SPDP 1998
CountryUnited States
CityOrlando
Period3/30/984/3/98

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Design and implementation of a parallel I/O runtime system for irregular applications'. Together they form a unique fingerprint.

Cite this