Automated tracing of I/O stack

Seong Jo Kim; Yuanrui Zhang; Seung Woo Son; Ramya Prabhakar; Mahmut Kandemir; Christina Patrick; Wei Keng Liao; Alok Choudhary

doi:10.1007/978-3-642-15646-5_8

Automated tracing of I/O stack

Seong Jo Kim, Yuanrui Zhang, Seung Woo Son, Ramya Prabhakar, Mahmut Kandemir, Christina Patrick, Wei Keng Liao, Alok Choudhary

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Scopus citations

Abstract

Efficient execution of parallel scientific applications requires high-performance storage systems designed to meet their I/O requirements. Most high-performance I/O intensive applications access multiple layers of the storage stack during their disk operations. A typical I/O request from these applications may include accesses to high-level libraries such as MPI I/O, executing on clustered parallel file systems like PVFS2, which are in turn supported by native file systems like Linux. In order to design and implement parallel applications that exercise this I/O stack, it is important to understand the flow of I/O calls through the entire storage system. Such understanding helps in identifying the potential performance and power bottlenecks in different layers of the storage hierarchy. To trace the execution of the I/O calls and to understand the complex interactions of multiple user-libraries and file systems, we propose an automatic code instrumentation technique, which enables us to collect detailed statistics of the I/O stack. Our proposed I/O tracing tool traces the flow of I/O calls across different layers of an I/O stack, and can be configured to work with different file systems and user-libraries. It also analyzes the collected information to generate output in terms of different user-specified metrics of interest.

Original language	English (US)
Title of host publication	Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings
Pages	72-81
Number of pages	10
DOIs	https://doi.org/10.1007/978-3-642-15646-5_8
State	Published - 2010
Event	17th European MPI Users' Group Meeting, EuroMPI 2010 - Stuttgart, Germany Duration: Sep 12 2010 → Sep 15 2010

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	6305 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	17th European MPI Users' Group Meeting, EuroMPI 2010
Country/Territory	Germany
City	Stuttgart
Period	9/12/10 → 9/15/10

Keywords

Automated code instrumentation
MPI-IO
MPICH2
PVFS2
Parallel I/O

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-642-15646-5_8

Cite this

Kim, S. J., Zhang, Y., Son, S. W., Prabhakar, R., Kandemir, M., Patrick, C., Liao, W. K., & Choudhary, A. (2010). Automated tracing of I/O stack. In Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings (pp. 72-81). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6305 LNCS). https://doi.org/10.1007/978-3-642-15646-5_8

@inproceedings{3e7a21d01c4046edab7102a1e938b5ab,

title = "Automated tracing of I/O stack",

abstract = "Efficient execution of parallel scientific applications requires high-performance storage systems designed to meet their I/O requirements. Most high-performance I/O intensive applications access multiple layers of the storage stack during their disk operations. A typical I/O request from these applications may include accesses to high-level libraries such as MPI I/O, executing on clustered parallel file systems like PVFS2, which are in turn supported by native file systems like Linux. In order to design and implement parallel applications that exercise this I/O stack, it is important to understand the flow of I/O calls through the entire storage system. Such understanding helps in identifying the potential performance and power bottlenecks in different layers of the storage hierarchy. To trace the execution of the I/O calls and to understand the complex interactions of multiple user-libraries and file systems, we propose an automatic code instrumentation technique, which enables us to collect detailed statistics of the I/O stack. Our proposed I/O tracing tool traces the flow of I/O calls across different layers of an I/O stack, and can be configured to work with different file systems and user-libraries. It also analyzes the collected information to generate output in terms of different user-specified metrics of interest.",

keywords = "Automated code instrumentation, MPI-IO, MPICH2, PVFS2, Parallel I/O",

author = "Kim, {Seong Jo} and Yuanrui Zhang and Son, {Seung Woo} and Ramya Prabhakar and Mahmut Kandemir and Christina Patrick and Liao, {Wei Keng} and Alok Choudhary",

note = "Funding Information: This work is supported in part by NSF grants 0937949, 0621402, 0724599, 0821527, 0833126, 0720749, 0621443, 0724599, and 0833131 and DOE grants DEAC02-06CH11357, DE-FG02-08ER25848, DE-SC0002156, and DESC0001283.; 17th European MPI Users' Group Meeting, EuroMPI 2010 ; Conference date: 12-09-2010 Through 15-09-2010",

year = "2010",

doi = "10.1007/978-3-642-15646-5_8",

language = "English (US)",

isbn = "3642156452",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "72--81",

booktitle = "Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings",

}

Kim, SJ, Zhang, Y, Son, SW, Prabhakar, R, Kandemir, M, Patrick, C, Liao, WK & Choudhary, A 2010, Automated tracing of I/O stack. in Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6305 LNCS, pp. 72-81, 17th European MPI Users' Group Meeting, EuroMPI 2010, Stuttgart, Germany, 9/12/10. https://doi.org/10.1007/978-3-642-15646-5_8

Automated tracing of I/O stack. / Kim, Seong Jo; Zhang, Yuanrui; Son, Seung Woo et al.

Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings. 2010. p. 72-81 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6305 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Automated tracing of I/O stack

AU - Kim, Seong Jo

AU - Zhang, Yuanrui

AU - Son, Seung Woo

AU - Prabhakar, Ramya

AU - Kandemir, Mahmut

AU - Patrick, Christina

AU - Liao, Wei Keng

AU - Choudhary, Alok

N1 - Funding Information: This work is supported in part by NSF grants 0937949, 0621402, 0724599, 0821527, 0833126, 0720749, 0621443, 0724599, and 0833131 and DOE grants DEAC02-06CH11357, DE-FG02-08ER25848, DE-SC0002156, and DESC0001283.

PY - 2010

Y1 - 2010

N2 - Efficient execution of parallel scientific applications requires high-performance storage systems designed to meet their I/O requirements. Most high-performance I/O intensive applications access multiple layers of the storage stack during their disk operations. A typical I/O request from these applications may include accesses to high-level libraries such as MPI I/O, executing on clustered parallel file systems like PVFS2, which are in turn supported by native file systems like Linux. In order to design and implement parallel applications that exercise this I/O stack, it is important to understand the flow of I/O calls through the entire storage system. Such understanding helps in identifying the potential performance and power bottlenecks in different layers of the storage hierarchy. To trace the execution of the I/O calls and to understand the complex interactions of multiple user-libraries and file systems, we propose an automatic code instrumentation technique, which enables us to collect detailed statistics of the I/O stack. Our proposed I/O tracing tool traces the flow of I/O calls across different layers of an I/O stack, and can be configured to work with different file systems and user-libraries. It also analyzes the collected information to generate output in terms of different user-specified metrics of interest.

AB - Efficient execution of parallel scientific applications requires high-performance storage systems designed to meet their I/O requirements. Most high-performance I/O intensive applications access multiple layers of the storage stack during their disk operations. A typical I/O request from these applications may include accesses to high-level libraries such as MPI I/O, executing on clustered parallel file systems like PVFS2, which are in turn supported by native file systems like Linux. In order to design and implement parallel applications that exercise this I/O stack, it is important to understand the flow of I/O calls through the entire storage system. Such understanding helps in identifying the potential performance and power bottlenecks in different layers of the storage hierarchy. To trace the execution of the I/O calls and to understand the complex interactions of multiple user-libraries and file systems, we propose an automatic code instrumentation technique, which enables us to collect detailed statistics of the I/O stack. Our proposed I/O tracing tool traces the flow of I/O calls across different layers of an I/O stack, and can be configured to work with different file systems and user-libraries. It also analyzes the collected information to generate output in terms of different user-specified metrics of interest.

KW - Automated code instrumentation

KW - MPI-IO

KW - MPICH2

KW - PVFS2

KW - Parallel I/O

UR - http://www.scopus.com/inward/record.url?scp=78149245030&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78149245030&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-15646-5_8

DO - 10.1007/978-3-642-15646-5_8

M3 - Conference contribution

AN - SCOPUS:78149245030

SN - 3642156452

SN - 9783642156458

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 72

EP - 81

BT - Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings

T2 - 17th European MPI Users' Group Meeting, EuroMPI 2010

Y2 - 12 September 2010 through 15 September 2010

ER -

Kim SJ, Zhang Y, Son SW, Prabhakar R, Kandemir M, Patrick C et al. Automated tracing of I/O stack. In Recent Advances in the Message Passing Interface - 17th European MPI Users' Group Meeting, EuroMPI 2010, Proceedings. 2010. p. 72-81. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-15646-5_8

Automated tracing of I/O stack

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this