TY - GEN
T1 - MS-I/O
T2 - 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGrid 2002
AU - Shen, Xiaohui
AU - Choudhary, Alok
PY - 2002
Y1 - 2002
N2 - More and more parallel applications are running in a distributed environment to take advantage of easily available and inexpensive commodity resources. For data intensive applications, employing multiple distributed storage resources has many advantages. In this paper, we present a Multi-Storage I/O System (MS-I/O) that can not only effectively manage various distributed storage resources in the system, but also provide novel high performance storage access schemes. MS-I/O employs many state-of-the-art I/O optimizations such as collective I/O, asynchronous I/O etc. and a number of new techniques such as data location, data replication, subfile, superfile and data access history. In addition, many MS-I/O optimization schemes can work simultaneously within a single data access session, greatly improving the performance. Although I/O optimization techniques can help improve performance, it also complicates I/O system. In addition, most optimization techniques have their limitations. Therefore, selecting accurate optimization policies requires ex-pert knowledge which is not suitable for end users who may have little knowledge of I/O techniques. So the task of I/O optimization decision should be left to the I/O system itself, that is, automatic from user's point of view. We present a User Access Pattern data structure which is associated With each dataset that can help MS-I/O easily make accurate I/O optimization decisions.
AB - More and more parallel applications are running in a distributed environment to take advantage of easily available and inexpensive commodity resources. For data intensive applications, employing multiple distributed storage resources has many advantages. In this paper, we present a Multi-Storage I/O System (MS-I/O) that can not only effectively manage various distributed storage resources in the system, but also provide novel high performance storage access schemes. MS-I/O employs many state-of-the-art I/O optimizations such as collective I/O, asynchronous I/O etc. and a number of new techniques such as data location, data replication, subfile, superfile and data access history. In addition, many MS-I/O optimization schemes can work simultaneously within a single data access session, greatly improving the performance. Although I/O optimization techniques can help improve performance, it also complicates I/O system. In addition, most optimization techniques have their limitations. Therefore, selecting accurate optimization policies requires ex-pert knowledge which is not suitable for end users who may have little knowledge of I/O techniques. So the task of I/O optimization decision should be left to the I/O system itself, that is, automatic from user's point of view. We present a User Access Pattern data structure which is associated With each dataset that can help MS-I/O easily make accurate I/O optimization decisions.
UR - http://www.scopus.com/inward/record.url?scp=84887905147&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84887905147&partnerID=8YFLogxK
U2 - 10.1109/CCGRID.2002.1017124
DO - 10.1109/CCGRID.2002.1017124
M3 - Conference contribution
AN - SCOPUS:84887905147
SN - 0769515827
SN - 9780769515823
T3 - 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGrid 2002
SP - 154
EP - 163
BT - 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGrid 2002
PB - IEEE Computer Society
Y2 - 21 May 2002 through 24 May 2002
ER -