COMPASSION: A parallel I/O runtime system including chunking and compression for irregular applications

Jesús Carretero, Jaechun No, Sung Soon Park, Alok Choudhary, Pang Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations

Abstract

In this paper we present two designs, namely, "Collective I/O" and "Pipelined Collective I/O", of a runtime library for irregular applications based on the two-phase collective I/O technique. We also present the optimization of both models by using chunking and compression mechanisms. In the first scheme, all processors participate in compressions and I/O at the same time, making scheduling of I/O requests simpler but creating a possibility of contention at the I/O nodes. In the second approach, processors are grouped into several groups, overlapping communication, compression, and I/O to reduce I/O contention dynamically. Finally, evaluation results are shown that demonstrates that we can obtain significantly high-performance for I/O above what has been possible so far.

Original languageEnglish (US)
Title of host publicationHigh-Performance Computing and Networking - International Conference and Exhibition, 1998, Proceedings
EditorsPeter Sloot, Bob Hertzberger, Marian Bubak
PublisherSpringer Verlag
Pages668-677
Number of pages10
ISBN (Print)3540644431, 9783540644439
DOIs
StatePublished - 1998
EventInternational Conference and Exhibition on High-Performance Computing and Networking, HPCN 1998 - Amsterdam, Netherlands
Duration: Apr 21 1998Apr 23 1998

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume1401
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherInternational Conference and Exhibition on High-Performance Computing and Networking, HPCN 1998
Country/TerritoryNetherlands
CityAmsterdam
Period4/21/984/23/98

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'COMPASSION: A parallel I/O runtime system including chunking and compression for irregular applications'. Together they form a unique fingerprint.

Cite this