Data access reorganizations in compiling out-of-core data parallel programs on distributed memory machines

Mahmut Kandemir*, Rajesh Bordawekar, Alok Choudhary

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

8 Scopus citations

Abstract

This paper describes optimization techniques for translating out-of-core programs written in a data parallel language to message passing node programs with explicit parallel I/O. We demonstrate that straightforward extension of in-core compilation techniques does not work well for out-of-core programs. We then describe how the compiler can optimize the code by (1) determining appropriate file layouts for out-of-core arrays, (2) permuting the loops in the nest(s) to allow efficient file access, and (3) partitioning the available node memory among references based on I/O cost estimation. Our experimental results indicate that these optimizations can reduce the amount of time spent in I/O by as much as an order of magnitude.

Original languageEnglish (US)
Pages (from-to)559-564
Number of pages6
JournalProceedings of the International Parallel Processing Symposium, IPPS
StatePublished - Jan 1 1997
EventProceedings of the 1997 11th International Parallel Processing Symposium, IPPS 97 - Geneva, Switz
Duration: Apr 1 1997Apr 5 1997

ASJC Scopus subject areas

  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Data access reorganizations in compiling out-of-core data parallel programs on distributed memory machines'. Together they form a unique fingerprint.

Cite this