Global I/O optimizations for out-of-core computations

Mahmut Kandemir*, Meena Kandaswamy, Alok Choudhary

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

The use of parallel machines to solve large scale computational problems in science and engineering has increased considerably in recent times. Many of these problems have computational requirements which stretch the capabilities of even the fastest machine available today. In addition to requiring a great deal of computational power, these problems usually deal with large quantities of data up to a few terabytes. The main memory sizes of current parallel machines do not even come close to matching these requirements; hence data needs to be stored on disks and fetched during the execution of the program. Unfortunately, current optimizing compilers for parallel machines provide support only for in-core computations in which the data sets can fit into memory. This limitation severely affects the performance of programs which depend on disk resident data. Our previous research demonstrated that file layout optimizations are extremely important for optimizing such programs. In this paper we investigate solutions to the global I/O optimization problem for out-of-core computations. Since the general problem is NP-complete, we present fast heuristics that can result in near-optimal solutions for the programs encountered in practice. Preliminary results provide encouraging evidence that our algorithms can be successful in optimizing out-of-core programs.

Original languageEnglish (US)
Title of host publicationProceedings of the International Conference on High Performance Computing, HiPC
Editors Anon
PublisherIEEE
Pages401-406
Number of pages6
StatePublished - Dec 1 1997
EventProceedings of the 1997 4th International Conference on High Performance Computing, HiPC - Bangalore, India
Duration: Dec 18 1997Dec 21 1997

Other

OtherProceedings of the 1997 4th International Conference on High Performance Computing, HiPC
CityBangalore, India
Period12/18/9712/21/97

ASJC Scopus subject areas

  • Computer Science(all)

Fingerprint Dive into the research topics of 'Global I/O optimizations for out-of-core computations'. Together they form a unique fingerprint.

Cite this