Toward Credible Evaluation of Anomaly-Based Intrusion-Detection Methods
- 17 May 2010
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews)
- Vol. 40 (5), 516-524
- https://doi.org/10.1109/tsmcc.2010.2048428
Abstract
Since the first introduction of anomaly-based intrusion detection to the research community in 1987, the field has grown tremendously. A variety of methods and techniques introducing new capabilities in detecting novel attacks were developed. Most of these techniques report a high detection rate of 98% at the low false alarm rate of 1%. In spite of the anomaly-based approach's appeal, the industry generally favors signature-based detection for mainstream implementation of intrusion-detection systems. While a variety of anomaly-detection techniques have been proposed, adequate comparison of these methods' strengths and limitations that can lead to potential commercial application is difficult. Since the validity of experimental research in academic computer science, in general, is questionable, it is plausible to assume that research in anomaly detection shares the above problem. The concerns about the validity of these methods may partially explain why anomaly-based intrusion-detection methods are not adopted by industry. To investigate this issue, we review the current state of the experimental practice in the area of anomaly-based intrusion detection and survey 276 studies in this area published during the period of 2000-2008. We summarize our observations and identify the common pitfalls among surveyed works.Keywords
This publication has 20 references indexed in Scilit:
- Empirical evaluation in Computer Science research published by ACMInformation and Software Technology, 2009
- The need for simulation in evaluating anomaly detectorsACM SIGCOMM Computer Communication Review, 2008
- On the success of empirical studies in the international conference on software engineeringPublished by Association for Computing Machinery (ACM) ,2006
- MANET simulation studiesACM SIGMOBILE Mobile Computing and Communications Review, 2005
- A Comparative Study of Anomaly Detection Schemes in Network Intrusion DetectionPublished by Society for Industrial & Applied Mathematics (SIAM) ,2003
- An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly DetectionLecture Notes in Computer Science, 2003
- On credibility of simulation studies of telecommunication networksIEEE Communications Magazine, 2002
- The 1998 Lincoln Laboratory IDS EvaluationLecture Notes in Computer Science, 2000
- Experimental evaluation in computer science: A quantitative studyJournal of Systems and Software, 1995
- ACM President's Letter: What is experimental computer science?Communications of the ACM, 1980