IDGraphs: Intrusion detection and analysis using stream compositing

Traffic anomalies and attacks are common-place in today's networks and identifying them rapidly and accurately is critical for operators of large networks. For a statistical intrusion detection system (IDS), it's crucial to detect at the flow-level. However, existing IDS systems offer only limited support for interactively examining detected intrusions and anomalies, analyzing worm propagation patterns, and discovering correlated attacks. These problems are becoming even more acute as the traffic on today's high-speed routers continues to grow. IDGraphs is an interactive visualization system for intrusion detection that addresses these challenges. The central visualization in the system is a flow-level trace plotted with time on the horizontal axis and the total number of unsuccessful connections (indicating suspicious traffic) on the vertical axis. The article summarizes a stack of tens or hundreds of thousands of these traces using the histographs technique, which composites the traces and maps data density at each pixel to brightness. Users can zoom into or interactively query the summary view, performing analysis by highlighting subsets of the traces. For example, brushing a linked correlation matrix view highlights traces with similar patterns, revealing distributed attacks that are difficult to detect using standard statistical analysis. The article discusses the application of IDGraphs to a real network router data set with millions of flow-level records representing total traffic in the terabyte range. The system successfully detects and analyzes a variety of attacks and anomalies, including port scanning, worm outbreaks, stealthy TCP SYN flooding, and some distributed attacks.