Scalable I/O and analytics

Alok Choudhary*, Wei Keng Liao, Kui Gao, Arifa Nisar, Robert Ross, Rajeev Thakur, Robert Latham

*Corresponding author for this work

Research output: Contribution to journalConference article

25 Scopus citations

Abstract

High-performance computing systems have already approached peta-scale with hundreds of thousands of processors/cores in many deployments. These systems promise a new level of predictive and knowledge discovery ability as researchers gain the capability to model dependencies between phenomena at scales not seen earlier. These applications are highly I/O and data intensive, leading scientists to observe that performing I/O and subsequent analyses are major bottlenecks in effectively utilizing peta-scale systems and a major hurdle in accelerating discoveries. Although significant progress has been made in performance, interfaces, and middleware runtime systems for I/O in the recent past, significantly more research and development needs to be carried out to scale the performance to the desired levels for systems containing tens to hundreds of thousands of cores. In this work we outline our recent achievements and current research for designing scalable I/O software and enabling data analytics in storage systems. We also enumerate key challenges for the I/O systems and discuss ongoing efforts that address these challenges.

Original languageEnglish (US)
Article number012048
JournalJournal of Physics: Conference Series
Volume180
Issue number1
DOIs
StatePublished - 2009

ASJC Scopus subject areas

  • Physics and Astronomy(all)

Fingerprint Dive into the research topics of 'Scalable I/O and analytics'. Together they form a unique fingerprint.

Cite this