TY - GEN
T1 - Optimizing I/O for irregular applications on distributed-memory machines
AU - Carretero, Jesús
AU - No, Jaechun
AU - Choudhary, Alok
N1 - Publisher Copyright:
© Springer-Verlag Berlin Heidelberg 1999.
PY - 1999
Y1 - 1999
N2 - In this paper we present the design, implementation and evaluation of a runtime system based on collective I/O techniques for irregular applications. Its main goal is to provide parallel collective I/O, by having all processors participate in the I/O simultaneously, and making the mapping of the I/O requests simpler. Using such a technique, the input/output of the irregular applications can be greatly simplified by always maintaining global files canonically ordered, thus avoiding the utilization of multiple files and the associated sorting/merging steps. The run-time library has been optimized by applying in-memory compression mechanisms to the collective I/O operations. We also present the results of several evaluation experiments obtained by running a particle in cell application on an Intel Paragon machine. Those results demonstrate that significantly high-performance for I/O can be obtained by using our library.
AB - In this paper we present the design, implementation and evaluation of a runtime system based on collective I/O techniques for irregular applications. Its main goal is to provide parallel collective I/O, by having all processors participate in the I/O simultaneously, and making the mapping of the I/O requests simpler. Using such a technique, the input/output of the irregular applications can be greatly simplified by always maintaining global files canonically ordered, thus avoiding the utilization of multiple files and the associated sorting/merging steps. The run-time library has been optimized by applying in-memory compression mechanisms to the collective I/O operations. We also present the results of several evaluation experiments obtained by running a particle in cell application on an Intel Paragon machine. Those results demonstrate that significantly high-performance for I/O can be obtained by using our library.
UR - http://www.scopus.com/inward/record.url?scp=84957701662&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84957701662&partnerID=8YFLogxK
U2 - 10.1007/3-540-49164-3_45
DO - 10.1007/3-540-49164-3_45
M3 - Conference contribution
AN - SCOPUS:84957701662
SN - 3540656413
SN - 9783540656418
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 470
EP - 479
BT - Parallel Computation - 4th International ACPC Conference Including Special Tracks on Parallel Numerics (ParNum 1999) and Parallel Computing in Image Processing, Video Processing, and Multimedia, Proceedings
A2 - Zinterhof, Peter
A2 - Vajteršic, Marian
A2 - Uhl, Andreas
PB - Springer Verlag
T2 - 4th International ACPC Conference on Parallel Computation, ACPC 1999
Y2 - 16 February 1999 through 18 February 1999
ER -