EARLY DETECTION OF NETWORK INTRUSIONS USING MACHINE LEARNING TECHNIQUES: A SYSTEMATIC LITERATURE REVIEW
DOI:
https://doi.org/10.30572/2018/KJE/170437Keywords:
Intrusions Detection, Anomaly Detection, Machine Learning, Early Detection, Supervised, Unsupervised, real-time detectionAbstract
The rising complexity and sophistication of threats in network traffic pose significant challenges that require precise and reliable Intrusion Detection Systems (IDSs). In recent years, these systems have gained considerable attention due to their continuously improving performance. The early detection of breaches is critical to limit potential damage, and Machine Learning (ML) methods have shown efficacy in improving IDSs. This review considers 30 recent research papers identified through a systematic screening process. This review seeks to thoroughly analyze and synthesize contemporary research on the classification of supervised and unsupervised ML approaches for the early detection of network breaches. The primary purpose of this review is to provide a thorough, multi-dimensional analysis of approaches for the early detection of intrusions in IDSs. The study emphasizes that early detection relies not only on high accuracy but also on computational efficiency and low latency. Accordingly, it analyzes the influence of feature selection on model effectiveness and speed, methods for handling data imbalance, and commonly used datasets and evaluation metrics. Our findings reveal that Random Forest (RF) and Extreme Gradient Boosting (XGBoost), in particular, demonstrate the optimal balance between high accuracy and low latency, making them superior for early detection compared to Deep Learning (DL) models which often incur higher computational costs. Our study helps researchers identify the most appropriate techniques for early detection of breaches using ML techniques, in addition to providing a research roadmap for future studies in this field
Downloads
References
Abdulganiyu, O.H., Ait Tchakoucht, T. and Saheed, Y.K. (2023) ‘A systematic literature review for network intrusion detection system (IDS)’, International Journal of Information Security, 22(5), pp. 1125–1162. Available at: https://doi.org/10.1007/s10207-023-00682-2.
Abualghanam, O. et al. (2023) ‘Real-Time Detection System for Data Exfiltration over DNS Tunneling Using Machine Learning’, Electronics (Switzerland), 12(6), pp. 1–21. Available at: https://doi.org/10.3390/electronics12061467.
Ahmad, Z. et al. (2021) ‘Network intrusion detection system: A systematic study of machine learning and deep learning approaches’, Transactions on Emerging Telecommunications Technologies, 32(1), pp. 1–29. Available at: https://doi.org/10.1002/ett.4150.
Aldhaheri, S. and Alhuzali, A. (2023) ‘SGAN-IDS: Self-Attention-Based Generative Adversarial Network against Intrusion Detection Systems’, Sensors, 23(18). Available at: https://doi.org/10.3390/s23187796.
Allagi, S., Pawan, T. and Leong, W.Y. (2025) ‘Enhanced Intrusion Detection Using Conditional-Tabular-Generative-Adversarial-Network-Augmented Data and a Convolutional Neural Network: A Robust Approach to Addressing Imbalanced Cybersecurity Datasets’, Mathematics, 13(12), p. 1923. Available at: https://doi.org/10.3390/math13121923.
Almaghthawi, Y., Ahmad, I. and Alsaadi, F.E. (2022) ‘Performance Analysis of Feature Subset Selection Techniques for Intrusion Detection’, Mathematics, 10(24). Available at: https://doi.org/10.3390/math10244745.
Almaiah, M.A. et al. (2022) ‘Performance Investigation of Principal Component Analysis for Intrusion Detection System Using Different Support Vector Machine Kernels.pdf’. Available at: https://doi.org/10.3390/electronics11213571.
AlSharman, S.A.D., Al-Khaleel, O. and Al-Ayyoub, M. (2024) ‘A Detailed Inspection of Machine Learning Based Intrusion Detection Systems for Software Defined Networks’, Internet of Things, 5(4), pp. 756–784. Available at: https://doi.org/10.3390/iot5040034.
Alzahrani, A.O. and Alenazi, M.J.F. (2021) ‘Designing a network intrusion detection system based on machine learning for software defined networks’, Future Internet, 13(5). Available at: https://doi.org/10.3390/fi13050111.
Balbal, K.F. (2024) ‘MIPART: A Partial Decision Tree-Based Method for Multiple-Instance Classification’, Applied Sciences (Switzerland), 14(24). Available at: https://doi.org/10.3390/app142411696.
Chae, H. et al. (2013) ‘Feature Selection for Intrusion Detection using NSL-KDD’, Recent Advances in Computer Science 20132, pp. 184–187.
Chen, K.H. et al. (2022) ‘Efficient Realization of Decision Trees for Real-Time Inference’, ACM Transactions on Embedded Computing Systems, 21(6). Available at: https://doi.org/10.1145/3508019.
Chen, T. and Guestrin, C. (2016) ‘XGBoost: A scalable tree boosting system’, Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 13-17-Augu, pp. 785–794. Available at: https://doi.org/10.1145/2939672.2939785.
Chen, Z. (1999) ‘ADASYN - Random Forest Based Intrusion Detection Model’, (April 2020), pp. 1–13. Available at: https://doi.org/10.48550/arXiv.2105.04301.
Chen, Z. et al. (2024) ‘Machine learning-enabled hybrid intrusion detection system with host data transformation and an advanced two-stage classifier’, Computer Networks, 250(April), p. 110576. Available at: https://doi.org/10.1016/j.comnet.2024.110576.
Dataset, K.D.D. (2025) ‘Perbandingan Algoritma Machine Learning untuk Intrusion Detection System pada Dataset NSL - KDD A Comparative Study of Machine Learning Algorithms for Intrusion Detection’, 14, pp. 1654–1664.
Dina, A.S., Siddique, A.B. and Manivannan, D. (2022) ‘Effect of Balancing Data Using Synthetic Data on the Performance of Machine Learning Classifiers for Intrusion Detection in Computer Networks’, IEEE Access, 10, pp. 96731–96747. Available at: https://doi.org/10.1109/ACCESS.2022.3205337.
Disha, R.A. and Waheed, S. (2022) ‘Performance analysis of machine learning models for intrusion detection system using Gini Impurity-based Weighted Random Forest (GIWRF) feature selection technique’, Cybersecurity, 5(1), pp. 1–22. Available at: https://doi.org/10.1186/s42400-021-00103-8.
Divekar, A. et al. (2018) ‘Benchmarking datasets for Anomaly-based Network Intrusion Detection: KDD CUP 99 alternatives’, Proceedings on 2018 IEEE 3rd International Conference on Computing, Communication and Security, ICCCS 2018, pp. 1–8. Available at: https://doi.org/10.1109/CCCS.2018.8586840.
Dumoulin, J. et al. (2018) ‘UNICITY: A depth maps database for people detection in security airlocks’, Proceedings of AVSS 2018 - 2018 15th IEEE International Conference on Advanced Video and Signal-Based Surveillance [Preprint]. Available at: https://doi.org/10.1109/AVSS.2018.8639152.
Elrawy, M.F., Awad, A.I. and Hamed, H.F.A. (2018) ‘Intrusion detection systems for IoT-based smart environments: a survey’, Journal of Cloud Computing, 7(1), pp. 1–20. Available at: https://doi.org/10.1186/s13677-018-0123-6.
Emirmahmutoğlu, E. and Atay, Y. (2025) ‘A feature selection-driven machine learning framework for anomaly-based intrusion detection systems’, Peer-to-Peer Networking and Applications, 18(3). Available at: https://doi.org/10.1007/s12083-025-01947-4.
García-Teodoro, P. et al. (2009) ‘Anomaly-based network intrusion detection: Techniques, systems and challenges’, Computers and Security, 28(1–2), pp. 18–28. Available at: https://doi.org/10.1016/j.cose.2008.08.003.
Gelman, B. et al. (no date) ‘That Escalated Quickly : An ML Framework for Alert Prioritization’. Available at: https://doi.org/10.48550/arXiv.2302.06648.
Ghadermazi, J., Shah, A. and Bastian, N.D. (2024) ‘Towards Real-time Network Intrusion Detection with Image-based Sequential Packets Representation’, IEEE Transactions on Big Data, 11(1), pp. 157–173. Available at: https://doi.org/10.1109/TBDATA.2024.3403394.
Giraddi, V. et al. (2024) ‘Machine Learning Approach to Intrusion Detection: Performance Evaluation’, Procedia Computer Science, 235, pp. 1851–1859. Available at: https://doi.org/10.1016/j.procs.2024.04.176.
Göcs, L. and Csaba, Z. (no date) ‘Identifying Relevant Features of CSE-CIC-IDS2018 Dataset for the Development of an Intrusion Detection System’, 0(0).
Goodfellow, I.J., Shlens, J. and Szegedy, C. (2015) ‘E XPLAINING AND H ARNESSING’, pp. 1–11. Available at: https://doi.org/10.48550/arXiv.1412.6572.
Guo, Y. (2023) ‘A review of Machine Learning-based zero-day attack detection: Challenges and future directions’, Computer Communications, 198(November 2022), pp. 175–185. Available at: https://doi.org/10.1016/j.comcom.2022.11.001.
Hu, Wei and Hu, Weiming (2005) ‘Network-based intrusion detection using adaboost algorithm’, Proceedings - 2005 IEEE/WIC/ACM InternationalConference on Web Intelligence, WI 2005, 2005, pp. 712–717. Available at: https://doi.org/10.1109/WI.2005.107.
Ige, T. and Kiekintveld, C. (2023) ‘Performance Comparison and Implementation of Bayesian Variants for Network Intrusion Detection’, 2023 IEEE International Conference on Artificial Intelligence, Blockchain, and Internet of Things, AIBThings 2023 - Proceedings, (15), pp. 1–5. Available at: https://doi.org/10.1109/AIBThings58340.2023.10292485.
Jin, Z. et al. (2020) ‘RFRSF: Employee Turnover Prediction Based on Random Forests and Survival Analysis’, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12343 LNCS, pp. 503–515. Available at: https://doi.org/10.1007/978-3-030-62008-0_35.
Kalchbrenner, N. et al. (2016) ‘G l s -t m’, pp. 1–15. Available at: https://doi.org/10.48550/arXiv.1507.01526.
Karamollaoğlu, H., Yücedağ, İ. and Doğru, İ.A. (2022) ‘A Hybrid PCA-MAO Based LSTM Model for Intrusion Detection in IoT Environments’, pp. 0–24.
Kim, A., Park, M. and Lee, D.H. (2020) ‘AI-IDS: Application of Deep Learning to Real-Time Web Intrusion Detection’, IEEE Access, 8, pp. 70245–70261. Available at: https://doi.org/10.1109/ACCESS.2020.2986882.
Kim, T. and Pak, W. (2021) ‘Hybrid Classification for High-Speed and High-Accuracy Network Intrusion Detection System’, IEEE Access, 9, pp. 83806–83817. Available at: https://doi.org/10.1109/ACCESS.2021.3087201.
Kim, T. and Pak, W. (2022) ‘Robust Network Intrusion Detection System Based on Machine-Learning With Early Classification’, IEEE Access, 10, pp. 10754–10767. Available at: https://doi.org/10.1109/ACCESS.2022.3145002.
Klumpner, C., Wijekoon, T. and Wheeler, P. (2006) ‘New methods for the active compensation of unbalanced supply voltages for two-stage direct power converters’, IEEJ Transactions on Industry Applications, 126(5), pp. 589–598. Available at: https://doi.org/10.1541/ieejias.126.589.
Kouchaksaraei, H.R. and Karl, H. (2019) ‘Service function chaining across openstack and kubernetes domains’, DEBS 2019 - Proceedings of the 13th ACM International Conference on Distributed and Event-Based Systems, pp. 240–243. Available at: https://doi.org/10.1145/3328905.3332505.
Lama, A. (2022) ‘A SURVEY ON NETWORK-BASED INTRUSION DETECTION SYSTEMS USING MACHINE’, 6(9), pp. 225–230.
Latif, Z. et al. (2022) ‘A Machine Learning-Based Anomaly Prediction Service for Software-Defined Networks’, Sensors, 22(21), pp. 1–17. Available at: https://doi.org/10.3390/s22218434.
Lee, J.H. and Park, K.H. (2019) ‘AE-CGAN model based high performance network intrusion detection system’, Applied Sciences (Switzerland), 9(20). Available at: https://doi.org/10.3390/app9204221.
Leela Jacob, S. and Parveen Sultana, H. (2025) ‘Spiking Residual ShuffleNet-Based Intrusion Detection in IoT Environment’, IEEE Access, 13(May), pp. 104257–104280. Available at: https://doi.org/10.1109/ACCESS.2025.3579777.
Logeswari, G. et al. (2025) ‘An improved synergistic dual-layer feature selection algorithm with two type classifier for efficient intrusion detection in IoT environment’, Scientific Reports, 15(1). Available at: https://doi.org/10.1038/s41598-025-91663-z.
Lundberg, S.M. and Lee, S.I. (2017) ‘A unified approach to interpreting model predictions’, Advances in Neural Information Processing Systems, 2017-Decem(Section 3), pp. 4766–4775. Available at: https://doi.org/10.48550/arXiv.1705.07874.
Macfadyen, J.D. (2016) ‘Lime’, Mining Engineering, 68(7), pp. 58–59. Available at: https://doi.org/10.48550/arXiv.1602.04938.
Maseer, Z.K. et al. (2024) ‘Meta-analysis and systematic review for anomaly network intrusion detection systems: Detection methods, dataset, validation methodology, and challenges’, IET Networks, 13(5–6), pp. 339–376. Available at: https://doi.org/10.1049/ntw2.12128.
Meidan, Y. et al. (2018) ‘N-BaIoT-Network-based detection of IoT botnet attacks using deep autoencoders’, IEEE Pervasive Computing, 17(3), pp. 12–22. Available at: https://doi.org/10.1109/MPRV.2018.03367731.
Moustafa, N. and Slay, J. (2015) ‘UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set)’, 2015 Military Communications and Information Systems Conference, MilCIS 2015 - Proceedings [Preprint], (November). Available at: https://doi.org/10.1109/MilCIS.2015.7348942.
Musleh, D. et al. (2023) ‘Intrusion Detection System Using Feature Extraction with Machine Learning Algorithms in IoT’, Journal of Sensor and Actuator Networks, 12(2). Available at: https://doi.org/10.3390/jsan12020029.
Nabi, F. and Zhou, X. (2024) ‘Enhancing intrusion detection systems through dimensionality reduction: A comparative study of machine learning techniques for cyber security’, Cyber Security and Applications, 2(January). Available at: https://doi.org/10.1016/j.csa.2023.100033.
Nassif, A.B. et al. (2021) ‘Machine Learning for Anomaly Detection: A Systematic Review’, IEEE Access, 9, pp. 78658–78700. Available at: https://doi.org/10.1109/ACCESS.2021.3083060.
Nassreddine, G., Nassereddine, M. and Al-Khatib, O. (2025) ‘Ensemble Learning for Network Intrusion Detection Based on Correlation and Embedded Feature Selection Techniques’, Computers, 14(3), pp. 1–23. Available at: https://doi.org/10.3390/computers14030082.
Nie, L. et al. (2022) ‘Intrusion Detection for Secure Social Internet of Things Based on Collaborative Edge Computing: A Generative Adversarial Network-Based Approach’, IEEE Transactions on Computational Social Systems, 9(1), pp. 134–145. Available at: https://doi.org/10.1109/TCSS.2021.3063538.
Ozkan-Okay, M. et al. (2021) ‘A Comprehensive Systematic Literature Review on Intrusion Detection Systems’, IEEE Access, 9, pp. 157727–157760. Available at: https://doi.org/10.1109/ACCESS.2021.3129336.
Pandey, A.K. et al. (2023) ‘Generative Adversarial Network and Bayesian Optimization in Multi-class Support Vector Machine for Intrusion Detection System’, International Journal of Intelligent Engineering and Systems, 16(1), pp. 110–119. Available at: https://doi.org/10.22266/ijies2023.0228.10.
Papernot, N. et al. (2016) ‘The limitations of deep learning in adversarial settings’, Proceedings - 2016 IEEE European Symposium on Security and Privacy, EURO S and P 2016, pp. 372–387. Available at: https://doi.org/10.1109/EuroSP.2016.36.
Park, C. et al. (2023) ‘An Enhanced AI-Based Network Intrusion Detection System Using Generative Adversarial Networks’, IEEE Internet of Things Journal, 10(3), pp. 2330–2345. Available at: https://doi.org/10.1109/JIOT.2022.3211346.
Pinto, A., Herrera, L. and Donoso, Y. (2023) ‘Survey on Intrusion Detection Systems Based on Machine’, pp. 1–18.
Raman, S. et al. (2024) ‘SYN-GAN: A robust intrusion detection system using GAN-based synthetic data for IoT security’, Internet of Things (Netherlands), 26(May), p. 101212. Available at: https://doi.org/10.1016/j.iot.2024.101212.
Rao, Y.N. and Suresh Babu, K. (2023) ‘An Imbalanced Generative Adversarial Network-Based Approach for Network Intrusion Detection in an Imbalanced Dataset’, Sensors (Basel, Switzerland), 23(1). Available at: https://doi.org/10.3390/s23010550.
Reyes, A.A. et al. (2020) ‘A machine learning based two-stage wi-fi network intrusion detection system’, Electronics (Switzerland), 9(10), pp. 1–18. Available at: https://doi.org/10.3390/electronics9101689.
Robinson, A.E. (2020) ‘XENON1T observes tritium’, pp. 1–3. Available at: https://doi.org/10.48550/arXiv.2006.13278.
Sadia, H. et al. (2024) ‘Intrusion Detection System for Wireless Sensor Networks: A Machine Learning Based Approach’, IEEE Access, 12(March), pp. 52565–52582. Available at: https://doi.org/10.1109/ACCESS.2024.3380014.
Saranya, T. et al. (2020) ‘Performance Analysis of Machine Learning Algorithms in Intrusion Detection System: A Review’, Procedia Computer Science, 171(2019), pp. 1251–1260. Available at: https://doi.org/10.1016/j.procs.2020.04.133.
Sarker, I.H. et al. (2020) ‘Cybersecurity data science: an overview from machine learning perspective’, Journal of Big Data, 7(1). Available at: https://doi.org/10.1186/s40537-020-00318-5.
Seo, W. and Pak, W. (2021) ‘Real-Time Network Intrusion Prevention System Based on Hybrid Machine Learning’, IEEE Access, 9, pp. 46386–46397. Available at: https://doi.org/10.1109/ACCESS.2021.3066620.
Shaker, B.N., Al-musawi, B. and Hassan, M.F. (no date) ‘Explainable AI for Enhancing IDS Against Advanced Persistent Kill Chain’. Available at: https://doi.org/10.48550/arXiv.2506.07480.
Shhatha, A.M. and Alsaif, O.I. (2025) ‘Enhancing Cybersecurity Through Malware Detection Based on Machine Learning Technique’, Kufa Journal of Engineering, 16(3), pp. 82–100. Available at: https://doi.org/10.30572/2018/KJE/160306.
Sivalingan, H. (2024) ‘Cloud-Smart Surveillance: Enhancing Anomaly Detection in Video Streams With Df-Convlstm-Based Vae-Gan’, Kufa Journal of Engineering, 15(4), pp. 125–140. Available at: https://doi.org/10.30572/2018/KJE/150409.
Telikani, A. et al. (2024) ‘A Cost-Sensitive Machine Learning Model With Multitask Learning for Intrusion Detection in IoT’, IEEE Transactions on Industrial Informatics, 20(3), pp. 3880–3890. Available at: https://doi.org/10.1109/TII.2023.3314208.
Tommaso Zoppi and Andrea Ceccarelli (2021) ‘Prepare for Trouble and Make it Double Supervised – Unsupervised’, Journal of Network and Computer Applications, pp. 1–16.
Uhm, Y. and Pak, W. (2021) ‘Real-time network intrusion prevention system using incremental feature generation’, Computers, Materials and Continua, 70(1), pp. 1631–1648. Available at: https://doi.org/10.32604/cmc.2022.019667.
Vinayakumar, R. et al. (2019) ‘Deep Learning Approach for Intelligent Intrusion Detection System’, IEEE Access, 7, pp. 41525–41550. Available at: https://doi.org/10.1109/ACCESS.2019.2895334.
Vitorino, J. et al. (2022) ‘A Comparative Analysis of Machine Learning Techniques for IoT Intrusion Detection’, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 13291 LNCS, pp. 191–207. Available at: https://doi.org/10.1007/978-3-031-08147-7_13.
Westphal, C., Hailes, S. and Musolesi, M. (2025) ‘Feature Selection for Network Intrusion Detection’, pp. 1599–1610. Available at: https://doi.org/10.1145/3690624.3709339.
Wu, T. et al. (2022) ‘Intrusion detection system combined enhanced random forest with SMOTE algorithm’, Eurasip Journal on Advances in Signal Processing, 2022(1). Available at: https://doi.org/10.1186/s13634-022-00871-6.
Zhao, F. et al. (2018) ‘A filter feature selection algorithm based on mutual information for intrusion detection’, Applied Sciences (Switzerland), 8(9). Available at: https://doi.org/10.3390/app8091535.
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2026 Russell Jassim Alabedi, Aymen Hasan Alawadi

This work is licensed under a Creative Commons Attribution 4.0 International License.












