Towards a Reliable Comparison and Evaluation of Network Intrusion Detection Systems Based on Machine Learning Approaches
Open Access
- 4 March 2020
- journal article
- research article
- Published by MDPI AG in Applied Sciences
- Vol. 10 (5), 1775
- https://doi.org/10.3390/app10051775
Abstract
Presently, we are living in a hyper-connected world where millions of heterogeneous devices are continuously sharing information in different application contexts for wellness, improving communications, digital businesses, etc. However, the bigger the number of devices and connections are, the higher the risk of security threats in this scenario. To counteract against malicious behaviours and preserve essential security services, Network Intrusion Detection Systems (NIDSs) are the most widely used defence line in communications networks. Nevertheless, there is no standard methodology to evaluate and fairly compare NIDSs. Most of the proposals elude mentioning crucial steps regarding NIDSs validation that make their comparison hard or even impossible. This work firstly includes a comprehensive study of recent NIDSs based on machine learning approaches, concluding that almost all of them do not accomplish with what authors of this paper consider mandatory steps for a reliable comparison and evaluation of NIDSs. Secondly, a structured methodology is proposed and assessed on the UGR’16 dataset to test its suitability for addressing network attack detection problems. The guideline and steps recommended will definitively help the research community to fairly assess NIDSs, although the definitive framework is not a trivial task and, therefore, some extra effort should still be made to improve its understandability and usability further.This publication has 29 references indexed in Scilit:
- A novel statistical technique for intrusion detection systemsFuture Generation Computer Systems, 2018
- PCA-based multivariate statistical network monitoring for anomaly detectionComputers & Security, 2016
- Real time intrusion detection system for ultra-high-speed big data environmentsThe Journal of Supercomputing, 2016
- The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data setInformation Security Journal: A Global Perspective, 2016
- Feature Analysis, Evaluation and Comparisons of Classification Algorithms Based on Noisy Intrusion DatasetProcedia Computer Science, 2016
- An empirical comparison of botnet detection methodsComputers & Security, 2014
- Anomaly-based network intrusion detection: Techniques, systems and challengesComputers & Security, 2009
- A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter OptimizationJournal of Heuristics, 2008
- Random ForestsMachine Learning, 2001