Mining anomalies using traffic feature distributions
Top Cited Papers
- 22 August 2005
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM)
- Vol. 35 (4), 217-228
- https://doi.org/10.1145/1080091.1080118
Abstract
The increasing practicality of large-scale flow capture makes it possible to conceive of traffic analysis methods that detect and identify a large and diverse set of anomalies. However the challenge of effectively analyzing this massive data source for anomaly diagnosis is as yet unmet. We argue that the distributions of packet features (IP addresses and ports) observed in flow traces reveals both the presence and the structure of a wide range of anomalies. Using entropy as a summarization tool, we show that the analysis of feature distributions leads to significant advances on two fronts: (1) it enables highly sensitive detection of a wide range of anomalies, augmenting detections by volume-based methods, and (2) it enables automatic classification of anomalies via unsupervised learning. We show that using feature distributions, anomalies naturally fall into distinct and meaningful clusters. These clusters can be used to automatically classify anomalies and to uncover new anomaly types. We validate our claims on data from two backbone networks (Abilene and Geant) and conclude that feature distributions show promise as a key element of a fairly general network anomaly diagnosis framework.Keywords
This publication has 16 references indexed in Scilit:
- Profiling internet backbone trafficPublished by Association for Computing Machinery (ACM) ,2005
- Combining routing and traffic data for detection of IP forwarding anomaliesPublished by Association for Computing Machinery (ACM) ,2004
- Structural analysis of network traffic flowsACM SIGMETRICS Performance Evaluation Review, 2004
- Automatically inferring patterns of resource consumption in network trafficPublished by Association for Computing Machinery (ACM) ,2003
- A framework for classifying denial of service attacksPublished by Association for Computing Machinery (ACM) ,2003
- Flash crowds and denial of service attacksPublished by Association for Computing Machinery (ACM) ,2002
- Observed structure of addresses in IP trafficPublished by Association for Computing Machinery (ACM) ,2002
- A signal analysis of network traffic anomaliesPublished by Association for Computing Machinery (ACM) ,2002
- Deriving traffic demands for operational IP networks: methodology and experienceIEEE/ACM Transactions on Networking, 2001
- Control Procedures for Residuals Associated With Principal Component AnalysisTechnometrics, 1979