Botgraph: Large scale spamming botnet detection

Yao Zhao, Yinglian Xie, Fang Yu, Qifa Ke, Yuan Yu, Yan Chen, Eliot Gillum

Research output: Chapter in Book/Report/Conference proceedingConference contribution

23 Scopus citations

Abstract

Network security applications often require analyzing huge volumes of data to identify abnormal patterns or activities. The emergence of cloud-computing models opens up new opportunities to address this challenge by leveraging the power of parallel computing. In this paper, we design and implement a novel system called BotGraph to detect a new type of botnet spamming attacks targeting major Web email providers. BotGraph uncovers the correlations among botnet activities by constructing large user-user graphs and looking for tightly connected subgraph components. This enables us to identify stealthy botnet users that are hard to detect when viewed in isolation. To deal with the huge data volume, we implement BotGraph as a distributed application on a computer cluster, and explore a number of performance optimization techniques. Applying it to two months of Hotmail log containing over 500 million users, BotGraph successfully identified over 26 million botnet-created user accounts with a low false positive rate. The running time of constructing and analyzing a 220GB Hotmail log is around 1.5 hours with 240 machines. We believe both our graph-based approach and our implementations are generally applicable to a wide class of security applications for analyzing large datasets.

Original languageEnglish (US)
Title of host publicationProceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2009
PublisherUSENIX Association
Pages321-334
Number of pages14
ISBN (Electronic)9781931971676
StatePublished - Jan 1 2019
Event6th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2009 - Boston, United States
Duration: Apr 22 2009Apr 24 2009

Publication series

NameProceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2009

Conference

Conference6th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2009
CountryUnited States
CityBoston
Period4/22/094/24/09

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Control and Systems Engineering

Fingerprint Dive into the research topics of 'Botgraph: Large scale spamming botnet detection'. Together they form a unique fingerprint.

  • Cite this

    Zhao, Y., Xie, Y., Yu, F., Ke, Q., Yu, Y., Chen, Y., & Gillum, E. (2019). Botgraph: Large scale spamming botnet detection. In Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2009 (pp. 321-334). (Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2009). USENIX Association.