TY - GEN
T1 - Noncontiguous locking techniques for parallel file systems
AU - Ching, Avery
AU - Liao, Wei Keng
AU - Choudhary, Alok
AU - Ross, Robert
AU - Ward, Lee
PY - 2007
Y1 - 2007
N2 - Many parallel scientific applications use high-level I/O APIs that offer atomic I/O capabilities. Atomic I/O in current parallel file systems is often slow when multiple processes simultaneously access interleaved, shared files. Current atomic I/O solutions are not optimized for handling noncontiguous access patterns because current locking systems have a fixed file system block-based granularity and do not leverage high-level access pattern information. In this paper we present a hybrid lock protocol that takes advantage of new list and datatype byte-range lock description techniques to enable high performance atomic I/O operations for these challenging access patterns. We implement our scalable distributed lock manager (DLM) in the PVFS parallel file system and show that these techniques improve locking throughput over a naive noncontiguous locking approach by several orders of magnitude in an array of lock-only tests. Additionally, in two scientific I/O benchmarks, we show the benefits of avoiding false sharing with our byte-range granular DLM when compared against a block-based lock system implementation. (c) 2007 ACM.
AB - Many parallel scientific applications use high-level I/O APIs that offer atomic I/O capabilities. Atomic I/O in current parallel file systems is often slow when multiple processes simultaneously access interleaved, shared files. Current atomic I/O solutions are not optimized for handling noncontiguous access patterns because current locking systems have a fixed file system block-based granularity and do not leverage high-level access pattern information. In this paper we present a hybrid lock protocol that takes advantage of new list and datatype byte-range lock description techniques to enable high performance atomic I/O operations for these challenging access patterns. We implement our scalable distributed lock manager (DLM) in the PVFS parallel file system and show that these techniques improve locking throughput over a naive noncontiguous locking approach by several orders of magnitude in an array of lock-only tests. Additionally, in two scientific I/O benchmarks, we show the benefits of avoiding false sharing with our byte-range granular DLM when compared against a block-based lock system implementation. (c) 2007 ACM.
UR - http://www.scopus.com/inward/record.url?scp=56749165162&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=56749165162&partnerID=8YFLogxK
U2 - 10.1145/1362622.1362658
DO - 10.1145/1362622.1362658
M3 - Conference contribution
AN - SCOPUS:56749165162
SN - 9781595937643
T3 - Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, SC'07
BT - Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, SC'07
T2 - 2007 ACM/IEEE Conference on Supercomputing, SC'07
Y2 - 10 November 2007 through 16 November 2007
ER -