Minimal-overhead virtualization of a large scale supercomputer

John R. Lange*, Kevin Pedretti, Peter A Dinda, Chang Bae, Patrick G. Bridges, Philip Soltero, Alexander Merritt

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

15 Scopus citations


Virtualization has the potential to dramatically increase the usability and reliability of high performance computing (HPC) systems. However, this potential will remain unrealized unless overheads can be minimized. This is particularly challenging on large scale machines that run carefully crafted HPC OSes supporting tightlycoupled, parallel applications. In this paper, we show how careful use of hardware and VMM features enables the virtualization of a large-scale HPC system, specifically a Cray XT4 machine, with .5% overhead on key HPC applications, microbenchmarks, and guests at scales of up to 4096 nodes. We describe three techniques essential for achieving such low overhead: passthrough I/O, workload-sensitive selection of paging mechanisms, and carefully controlled preemption. These techniques are forms of symbiotic virtualization, an approach on which we elaborate.

Original languageEnglish (US)
Pages (from-to)169-180
Number of pages12
JournalACM SIGPLAN Notices
Issue number7
StatePublished - Jul 2011


  • High performance computing
  • Parallel computing
  • Virtual machine monitors

ASJC Scopus subject areas

  • General Computer Science


Dive into the research topics of 'Minimal-overhead virtualization of a large scale supercomputer'. Together they form a unique fingerprint.

Cite this