Noncontiguous locking techniques for parallel file systems

Avery Ching*, Wei Keng Liao, Alok Choudhary, Robert Ross, Lee Ward

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Many parallel scientific applications use high-level I/O APIs that offer atomic I/O capabilities. Atomic I/O in current parallel file systems is often slow when multiple processes simultaneously access interleaved, shared files. Current atomic I/O solutions are not optimized for handling noncontiguous access patterns because current locking systems have a fixed file system block-based granularity and do not leverage high-level access pattern information. In this paper we present a hybrid lock protocol that takes advantage of new list and datatype byte-range lock description techniques to enable high performance atomic I/O operations for these challenging access patterns. We implement our scalable distributed lock manager (DLM) in the PVFS parallel file system and show that these techniques improve locking throughput over a naive noncontiguous locking approach by several orders of magnitude in an array of lock-only tests. Additionally, in two scientific I/O benchmarks, we show the benefits of avoiding false sharing with our byte-range granular DLM when compared against a block-based lock system implementation. (c) 2007 ACM.

Original languageEnglish (US)
Title of host publicationProceedings of the 2007 ACM/IEEE Conference on Supercomputing, SC'07
DOIs
StatePublished - 2007
Event2007 ACM/IEEE Conference on Supercomputing, SC'07 - Reno, NV, United States
Duration: Nov 10 2007Nov 16 2007

Publication series

NameProceedings of the 2007 ACM/IEEE Conference on Supercomputing, SC'07

Other

Other2007 ACM/IEEE Conference on Supercomputing, SC'07
Country/TerritoryUnited States
CityReno, NV
Period11/10/0711/16/07

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Noncontiguous locking techniques for parallel file systems'. Together they form a unique fingerprint.

Cite this