A coordinated data collection approach: Design, evaluation, and comparison

William C. Cheng*, Cheng Fu Chou, Leana Golubchik, Samir Khuller, Yung Chun Wan

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

20 Scopus citations

Abstract

We consider the problem of collecting a large amount of data from several different hosts to a single destination in a wide-area network. This problem is important since improvements in data collection times in many applications such as wide-area upload applications, high-performance computing applications, and data mining applications are crucial to performance of those applications. Often, due to congestion conditions, the paths chosen by the network may have poor throughput. By choosing an alternate route at the application level, we may be able to obtain substantially faster completion time. This data collection problem is a nontrivial one because the issue is not only to avoid congested link(s), but to devise a coordinated transfer schedule which would afford maximum possible utilization of available network resources. Our approach for computing coordinated data collection schedules makes no assumptions about knowledge of the topology of the network or the capacity available on individual links of the network. This approach provides significant performance improvements under various degrees and types of network congestions. To show this, we give a comprehensive comparison study of the various approaches to the data collection problem which considers performance, robustness, and adaptation characteristics of the different data collection methods. The adaptation to network conditions characteristics are important as the above applications are long lasting, i.e., it is likely changes in network conditions will occur during the data transfer process. In general, our approach can be used for solving arbitrary data movement problems over the Internet. We use the Bistro platform to illustrate one application of our techniques.

Original languageEnglish (US)
Pages (from-to)2004-2018
Number of pages15
JournalIEEE Journal on Selected Areas in Communications
Volume22
Issue number10
DOIs
StatePublished - Dec 2004

Keywords

  • Data collection
  • Graph theory
  • Internet-based applications
  • Performance evaluation
  • System design

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'A coordinated data collection approach: Design, evaluation, and comparison'. Together they form a unique fingerprint.

Cite this