Design and evaluation of a compiler-directed collective I/O technique

Gokhan Memik, Mahmut T. Kandemir, Alok Nidhi Choudhary

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations


Current approaches to parallel I/O demand extensive user effort to obtain acceptable performance. This is in part due to difficulties in understanding the characteristics of a wide variety of I/O devices and in part due to inherent complexity of I/O software. While parallel I/O systems provide users with environments where large datasets can be shared between parallel processors, the ultimate performance of I/O-intensive codes depends largely on the relation between data access patterns and storage patterns of data in files and on disks. Collective I/O is one of the most popular methods to access the data when the storage and access patterns do not match. In this strategy, each processor does I/O on behalf of other processors if doing so improves the overall performance. While it is generally accepted that collective I/O and its variants can bring impressive improvements as far as the I/O performance is concerned, it is difficult for the programmer to use collective I/O effectively. In this paper, we propose and evaluate a compiler-directed collective I/O approach which detects the opportunities for collective I/O and inserts the necessary I/O calls in the code automatically. An important characteristic of the approach is that instead of applying collective I/O indiscriminately, it uses collective I/O selectively, only in cases where independent parallel I/O would not be possible. We have conducted several experiments using an IBM SP-2di stributed-memory message-passing machine with 128 nodes. Our compiler directed collective I/O scheme was able to perform 18% better in average than an indiscriminate collective I/O scheme in our base configuration.

Original languageEnglish (US)
Title of host publicationEuro-Par 2000 Parallel Processing - 6th International Euro-Par Conference, Proceedings
EditorsArndt Bode, Thomas Ludwig, Wolfgang Karl, Roland Wismüller
PublisherSpringer Verlag
Number of pages10
ISBN (Electronic)9783540679561
StatePublished - 2000
Event6th International European Conference on Parallel Computing, Euro-Par 2000 - Munich, Germany
Duration: Aug 29 2000Sep 1 2000

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other6th International European Conference on Parallel Computing, Euro-Par 2000

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Design and evaluation of a compiler-directed collective I/O technique'. Together they form a unique fingerprint.

Cite this