Large-scale data collection: A coordinated approach

William C. Cheng*, Cheng Fu Chou, Leana Golubchik, Samir Khuller, Yung Chun Wan

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

18 Scopus citations

Abstract

In this paper we consider the problem of collecting a large amount of data from several different hosts to a single destination in a wide-area network. Often, due to congestion conditions, the paths chosen by the network may have poor throughput. By choosing an alternate route at the application level, we may be able to obtain substantially faster completion time. This data collection problem is a non-trivial one because the issue is not only to avoid congested link(s), but to devise a coordinated transfer schedule which would afford maximum possible utilization of available network resources. In this paper we present an approach for computing coordinated data collection schedules, which can result in significant performance improvements. We make no assumptions about knowledge of the topology of the network or the capacity available on individual links of the network, i.e., we only use end-to-end information. Finally, we also study the shortcomings of this approach in terms of the gap between the theoretical formulation and the resulting data transfers in wide-area networks. In general, our approach can be used for solving arbitrary data movement problems over the Internet. We use the Bistro platform to illustrate one application of our techniques.

Original languageEnglish (US)
Pages (from-to)218-228
Number of pages11
JournalProceedings - IEEE INFOCOM
Volume1
StatePublished - 2003
Event22nd Annual Joint Conference on the IEEE Computer and Communications Societies - San Francisco, CA, United States
Duration: Mar 30 2003Apr 3 2003

Keywords

  • Graph theory
  • Simulations
  • System design

ASJC Scopus subject areas

  • Computer Science(all)
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Large-scale data collection: A coordinated approach'. Together they form a unique fingerprint.

Cite this