Collective caching: Application-aware client-side file caching

Wei Keng Liao, Kenin Coloma, Alok Choudhary, Lee Ward, Eric Russell, Sonja Tideman

Research output: Contribution to journalConference articlepeer-review

52 Scopus citations


Parallel file subsystems in today's high-performance computers adopt many I/O optimization strategies that were designed for distributed systems. These strategies, for instance client-side file caching, treat each I/O request process independently, due to the consideration that clients are unlikely related with each other in a distributed environment. However, it is inadequate to apply such strategies directly in the high-performance computers where most of the I/O requests come from the processes that work on the same parallel applications. We believe that client-side caching could perform more effectively if the caching sub-system is aware of the process scope of an application and regards all the application processes as a single client. In this paper, we propose the idea of "collective caching" which coordinates the application processes to manage cache data and achieve cache coherence without involving the I/O servers. To demonstrate this idea, we implemented a collective caching sub-system at user space as a library, which can be incorporated into any Message Passing Interface implementation to increase its portability. The performance evaluation is presented with three I/O benchmarks on an IBM SP using it native parallel file system, GPFS. Our results show significant performance enhancement obtained by collective caching over the traditional approaches.

Original languageEnglish (US)
Pages (from-to)81-90
Number of pages10
JournalProceedings of the IEEE International Symposium on High Performance Distributed Computing
StatePublished - 2005
Event14th IEEE International Symposium on High Performance Distributed Computing, HPDC-14 - Research Triangle Park, NC, United States
Duration: Jul 24 2005Jul 27 2005

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications


Dive into the research topics of 'Collective caching: Application-aware client-side file caching'. Together they form a unique fingerprint.

Cite this