Diagnosing network-wide traffic anomalies
- 30 August 2004
- journal article
- Published by Association for Computing Machinery (ACM) in ACM SIGCOMM Computer Communication Review
- Vol. 34 (4), 219-230
- https://doi.org/10.1145/1030194.1015492
Abstract
Anomalies are unusual and significant changes in a network's traffic levels, which can often span multiple links. Diagnosing anomalies is critical for both network operators and end users. It is a difficult problem because one must extract and interpret anomalous patterns from large amounts of high-dimensional, noisy data.In this paper we propose a general method to diagnose anomalies. This method is based on a separation of the high-dimensional space occupied by a set of network traffic measurements into disjoint subspaces corresponding to normal and anomalous network conditions. We show that this separation can be performed effectively by Principal Component Analysis.Using only simple traffic measurements from links, we study volume anomalies and show that the method can: (1) accurately detect when a volume anomaly is occurring; (2) correctly identify the underlying origin-destination (OD) flow which is the source of the anomaly; and (3) accurately estimate the amount of traffic involved in the anomalous OD flow.We evaluate the method's ability to diagnose ( i.e., detect, identify, and quantify) both existing and synthetically injected volume anomalies in real traffic from two backbone networks. Our method consistently diagnoses the largest volume anomalies, and does so with a very low false alarm rate.Keywords
This publication has 12 references indexed in Scilit:
- Structural analysis of network traffic flowsPublished by Association for Computing Machinery (ACM) ,2004
- Combining routing and traffic data for detection of IP forwarding anomaliesPublished by Association for Computing Machinery (ACM) ,2004
- An information-theoretic approach to traffic matrix estimationPublished by Association for Computing Machinery (ACM) ,2003
- Sketch-based change detectionPublished by Association for Computing Machinery (ACM) ,2003
- Multivariate process monitoring and fault diagnosis by multi-scale PCAComputers & Chemical Engineering, 2002
- A signal analysis of network traffic anomaliesPublished by Association for Computing Machinery (ACM) ,2002
- Deriving traffic demands for operational IP networks: methodology and experienceIEEE/ACM Transactions on Networking, 2001
- Schemes for fault identification in communication networksIEEE/ACM Transactions on Networking, 1995
- A Singular Value Decomposition Updating Algorithm for Subspace TrackingSIAM Journal on Matrix Analysis and Applications, 1992
- Control Procedures for Residuals Associated With Principal Component AnalysisTechnometrics, 1979