A case study for scientific I/O: Improving the FLASH astrophysics code

Rob Latham*, Chris Daley, Wei Keng Liao, Kui Gao, Rob Ross, Anshu Dubey, Alok Choudhary

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

24 Scopus citations

Abstract

The FLASH code is a computational science tool for simulating and studying thermonuclear reactions. The program periodically outputs large checkpoint files (to resume a calculation from a particular point in time) and smaller plot files (for visualization and analysis). Initial experiments on BlueGene/P spent excessive time in input/output (I/O), making it difficult to do actual science. Our investigation of time spent in I/O revealed several locations in the I/O software stack where we could make improvements. Fixing data corruption in the MPI-IO library allowed us to use collective I/O, yielding an order of magnitude improvement. Restructuring the data layout provided a more efficient I/O access pattern and yielded another doubling of performance, but broke format assumptions made by other tools in the application workflow. Using new nonblocking APIs in the Parallel-NetCDF library allowed us to keep high performance and maintain backward compatibility. The I/O research community has studied a host of optimizations and strategies. Sometimes the challenge for applications is knowing how to apply these new techniques to production codes. In this case study, we offer a demonstration of how computational scientists, with a detailed understanding of their application, and the I/O community, with a wide array of approaches from which to choose, can magnify each other's efforts and achieve tremendous application productivity gains.

Original languageEnglish (US)
Article number015001
JournalComputational Science and Discovery
Volume5
Issue number1
DOIs
StatePublished - Jan 2012

ASJC Scopus subject areas

  • Numerical Analysis
  • Physics and Astronomy(all)
  • Computational Mathematics

Fingerprint Dive into the research topics of 'A case study for scientific I/O: Improving the FLASH astrophysics code'. Together they form a unique fingerprint.

Cite this