Fast VMM-based overlay networking for bridging the cloud and high performance computing

Lei Xia*, Zheng Cui, John Lange, Yuan Tang, Peter A Dinda, Patrick Bridges

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


A collection of virtual machines (VMs) interconnected with an overlay network with a layer 2 abstraction has proven to be a powerful, unifying abstraction for adaptive distributed and parallel computing on loosely-coupled environments. It is now feasible to allow VMs hosting high performance computing (HPC) applications to seamlessly bridge distributed cloud resources and tightly-coupled supercomputing and cluster resources. However, to achieve the application performance that the tightly-coupled resources are capable of, it is important that the overlay network not introduce significant overhead relative to the native hardware, which is not the case for current user-level tools, including our own existing VNET/U system. In response, we describe the design, implementation, and evaluation of a virtual networking system that has negligible latency and bandwidth overheads in 1-10 Gbps networks. Our system, VNET/P, is directly embedded into our publicly available Palacios virtual machine monitor (VMM). VNET/P achieves native performance on 1 Gbps Ethernet networks and very high performance on 10 Gbps Ethernet networks. The NAS benchmarks generally achieve over 95 % of their native performance on both 1 and 10 Gbps. We have further demonstrated that VNET/P can operate successfully over more specialized tightly-coupled networks, such as Infiniband and Cray Gemini. Our results suggest it is feasible to extend a software-based overlay network designed for computing at wide-area scales into tightly-coupled environments.

Original languageEnglish (US)
Pages (from-to)39-59
Number of pages21
JournalCluster Computing
Issue number1
StatePublished - Mar 2014


  • HPC
  • Overlay networks
  • Scalability
  • Virtualization

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Fast VMM-based overlay networking for bridging the cloud and high performance computing'. Together they form a unique fingerprint.

Cite this