Optimizing I/O for irregular applications on distributed-memory machines

Jesús Carretero, Jaechun No, Alok Choudhary

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper we present the design, implementation and evaluation of a runtime system based on collective I/O techniques for irregular applications. Its main goal is to provide parallel collective I/O, by having all processors participate in the I/O simultaneously, and making the mapping of the I/O requests simpler. Using such a technique, the input/output of the irregular applications can be greatly simplified by always maintaining global files canonically ordered, thus avoiding the utilization of multiple files and the associated sorting/merging steps. The run-time library has been optimized by applying in-memory compression mechanisms to the collective I/O operations. We also present the results of several evaluation experiments obtained by running a particle in cell application on an Intel Paragon machine. Those results demonstrate that significantly high-performance for I/O can be obtained by using our library.

Original languageEnglish (US)
Title of host publicationParallel Computation - 4th International ACPC Conference Including Special Tracks on Parallel Numerics (ParNum 1999) and Parallel Computing in Image Processing, Video Processing, and Multimedia, Proceedings
EditorsPeter Zinterhof, Marian Vajteršic, Andreas Uhl
PublisherSpringer Verlag
Pages470-479
Number of pages10
ISBN (Print)3540656413, 9783540656418
DOIs
StatePublished - 1999
Event4th International ACPC Conference on Parallel Computation, ACPC 1999 - Salzburg, Austria
Duration: Feb 16 1999Feb 18 1999

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume1557
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other4th International ACPC Conference on Parallel Computation, ACPC 1999
Country/TerritoryAustria
CitySalzburg
Period2/16/992/18/99

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'Optimizing I/O for irregular applications on distributed-memory machines'. Together they form a unique fingerprint.

Cite this