Communication strategies for out-of-core programs on distributed memory machines

Rajesh Bordawekar, Alok Choudhary

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

In this paper, we show that communication in the out-of-core distributed memory problems requires both inter-processor communication and file I/O. Thus, in order to improve I/O performance, it is necessary to optimize the I/O costs associated with a communication step. We present three methods for performing communication in out-of-core distributed memory problems. The first method, termed as the "out-of-core" communication method, follows a loosely synchronous model. Computation and Communication phases in this case are clearly separated, and communication requires permutation of data in files. The second method, termed as "demand-driven-in-core communication" considers only communication required of each in-core data slab individually. The third method, termed as "producer-driven-in-core communication" goes even one step further and tries to identify the potential (future) use of data while it is in memory. We describe these methods in detail and provide performance results for out-of-core applications; namely, two-dimensional FFT and two-dimensional elliptic solver. Finally, we discuss how "out-of-core" and "in-core" communication methods could be used in virtual memory environments on distributed memory machines.

Original languageEnglish (US)
Title of host publicationProceedings of the 9th International Conference on Supercomputing, ICS 1995
PublisherAssociation for Computing Machinery
Pages395-403
Number of pages9
ISBN (Electronic)0897917286
DOIs
StatePublished - Jul 3 1995
Event9th International Conference on Supercomputing, ICS 1995 - Barcelona, Spain
Duration: Jul 3 1995Jul 7 1995

Publication series

NameProceedings of the International Conference on Supercomputing
VolumePart F129361

Other

Other9th International Conference on Supercomputing, ICS 1995
CountrySpain
CityBarcelona
Period7/3/957/7/95

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Communication strategies for out-of-core programs on distributed memory machines'. Together they form a unique fingerprint.

Cite this