Performance Evaluation of Deep Learning Models on Diverse IoT Datasets for Intrusion Detection

Waqas, Abdullah

doi:10.22042/isecure.2025.507948.1208

Performance Evaluation of Deep Learning Models on Diverse IoT Datasets for Intrusion Detection

Document Type : Research Article

Author

Abdullah Waqas

Department of Electrical Engineering, National University of Technology, Islamabad, Pakistan

https://doi.org/10.22042/isecure.2025.507948.1208

Abstract

The Internet of Things (IoT) offers transformative potential across sectors like energy, defense, and healthcare, but its limited resources make it vulnerable to cyberattacks, necessitating robust security measures such as intrusion detection systems (IDS) to safeguard its infrastructure. This article presents a study that helps intrusion detection systems identify malicious and legitimate communications. To help the system make the best decisions possible, the subcategory of the attacked traffic is also classified. We trained the suggested models to be capable of binary and multi-class classification, targeting common attacks like denial of service (DoS), distributed denial of service (DDoS), reconnaissance, and information theft directed at IoT devices. Our methodology makes use of recently published IoT datasets, such as BoTIoT, ToNIoT, WUSTL-IIOT-20212021, and CiCIoT. To assess and contrast the performance of the proposed models on these datasets, we first applied stratified undersampling to convert the original imbalanced datasets into balanced subsets, which were then used for training and evaluation. Among the models evaluated, biLSTM achieved the highest accuracy of 99.66% and MCC of 0.99759 on the WUSTL-IIoT-2021 dataset. On the BoTIoT dataset, CNN with Dual Focal Loss reached 97.76% accuracy and 0.95536 MCC. For ToNIoT, LSTM achieved 97.01% accuracy with an MCC of 0.93643, while on the CiCIoT dataset, biLSTM obtained 96.23% accuracy and 0.96347 MCC. The results show that biLSTM and LSTM models give higher performance than FNN and CNN models in terms of precision, recall, F1 score, and MCC across all datasets, demonstrating improved performance for temporal IoT intrusion detection tasks.

Keywords

Deep Learning

Internet of Things

Neural Networks

Cybersecurity

[1] Kaspersky Lab. Kaspersky blocked over 330 thousand attacks on iot devices in the middle east in 2022, 2023. URL https://me-en.kaspersky.com/about/press-releases/kaspersky-blocked-over-330-thousand-attacks-on-iot-devices-in-the-middle-east-in-2022. Accessed: 2025-05-15.
[2] Alyazia Aldhaheri, Fatima Alwahedi, Mohamed Amine Ferrag, and Ammar Battah. Deep learning for cyber threat detection in iot networks: A review. Internet of Things and CyberPhysical Systems, 2023.
[3] Durgesh Srivastava, Rajeshwar Singh, Chinmay Chakraborty, Sunil Kr Maakar, Aaisha Makkar, and Deepak Sinwar. A framework for detection of cyber attacks by the classification of intrusion detection datasets. Microprocessors and Microsystems, 105:104964, 2024.
[4] Azar Abid Salih and Adnan Mohsin Abdulazeez. Evaluation of classification algorithms for intrusion detection system: A review. Journal of Soft Computing and Data Mining, 2(1):31–40, 2021.
[5] Umamah bint Khalid, Muddasar Naeem, Fabrizio Stasolla, Madiha Haider Syed, Musarat Abbas, and Antonio Coronato. Impact of ai-powered solutions in rehabilitation process: Recent improvements and future trends. International Journal of General Medicine, pages 943–969, 2024.
[6] Antonio Coronato, Muddasar Naeem, Giuseppe De Pietro, and Giovanni Paragliola. Reinforcement learning for intelligent healthcare applications: A survey. Artificial Intelligence in Medicine, 109:101964, 2020.
[7] Zahedi Azam, Md Motaharul Islam, and Mohammad Nurul Huda. Comparative analysis of intrusion detection systems and machine learning based model analysis through decision tree. IEEE Access, 2023.
[8] Francisco L Loaiza, John D Birdwell, George L Kennedy, and Dale Visser. Utility of artificial intelligence and machine learning in cybersecurity. JSTOR, 2022.
[9] Yanxia Sun and Zenghui Wang. Intrusion detection in iot and wireless networks using imagebased neural network classification. Applied SoftISeCure Computing, page 113236, 2025.
[10] Nathan Shone, Tran N Ngoc, Vu D Phai, and Qi Shi. A deep learning approach to network intrusion detection. IEEE Transactions on Emerging Topics in Computational Intelligence, 2(1): 41–50, 2018.
[11] Ahmad Javaid, Qussai Niyaz, Weiqing Sun, and Mansoor Alam. A deep learning approach for network intrusion detection system. In Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies (formerly BIONETICS), pages 21–26, 2016.
[12] Iman Sharafaldin, Arash Habibi Lashkari, Ali A Ghorbani, et al. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp, 1:108–116, 2018.
[13] Alberto Parmisano, Sebastian Garcia, and M Jose Erquiaga. A labeled dataset with malicious and benign iot network traffic. Stratosphere Laboratory: Praha, Czech Republic, 2020.
[14] Yahya Al-Hadhrami and Farookh Khadeer Hussain. Real time dataset generation framework for intrusion detection systems in iot. Future Generation Computer Systems, 108:414–423, 2020.
[15] Nour Moustafa and Jill Slay. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 Military Communications and Information Systems Conference (MilCIS), pages 1–6, 2015.
[16] Muder Almiani, Alia AbuGhazleh, Amer Al-Rahayfeh, Saleh Atiewi, and Abdul Razaque. Deep recurrent neural network for iot intrusion detection system. Simulation Modelling Practice and Theory, 101:102031, 2020.
[17] Rasheed Ahmad, Izzat Alsmadi, Wasim Alhamdani, and Lo’ai Tawalbeh. A comprehensive deep learning benchmark for iot ids. Computers & Security, 114:102588, 2022.
[18] Yazan Otoum, Dandan Liu, and Amiya Nayak. Dl-ids: a deep learning–based intrusion detection framework for securing iot. Transactions on Emerging Telecommunications Technologies, 33(3):e3803, 2022.
[19] Muhammad Almas Khan, Muazzam A Khan, Sana Ullah Jan, Jawad Ahmad, Sajjad Shaukat Jamal, Awais Aziz Shah, Nikolaos Pitropakis, and William J Buchanan. A deep learning-based intrusion detection system for mqtt enabled iot. Sensors, 21(21):7016, 2021.
[20] Bipraneel Roy and Hon Cheung. A deep learning approach for intrusion detection in internet of things using bidirectional long short-term memory recurrent neural network. In 2018 28th international telecommunication networks and applications conference (ITNAC), pages 1–6. IEEE, 2018.
[21] Hai-Viet Le, Quoc-Dung Ngo, and Van-Hoang Le. Iot botnet detection using system call graphs and one-class cnn classification. Int. J. Innov. Technol. Explor. Eng, 8(10):937–942, 2019.
[22] Ravi Vinayakumar, Mamoun Alazab, K Padannayil Soman, Prabaharan Poornachandran, Ameer Al-Nemrat, and Sitalakshmi Venkatraman. Deep learning approach for intelligent intrusion detection system. Ieee Access, 7:41525–41550, 2019.
[23] Meysam Vakili, Mohammad Ghamsari, and Masoumeh Rezaei. Performance analysis and comparison of machine and deep learning algorithms for iot data classification. arXiv preprint arXiv:2001.09636, 2020.
[24] Ahsan Nazir, Jingsha He, Nafei Zhu, Saima Siraj Qureshi, Siraj Uddin Qureshi, Faheem Ullah, Ahsan Wajahat, and Muhammad Salman Pathan. A deep learning-based novel hybrid cnn-lstm architecture for efficient detection of threats in the iot ecosystem. Ain Shams Engineering Journal, 15(7):102777, 2024.
[25] Meryem Amar and Bouabid El Ouahidi. A weighted lstm deep learning for intrusion detection. In International Conference on Advanced Communication Systems and Information Security, pages 170–179. Springer, 2019.
[26] Abdullah Alqahtani. Optimized deep autoencoder and bilstm for intrusion detection in iotsfog computing. Multimedia Tools and Applications, 84(8):4907–4943, 2025.
[27] Uday Chandra Akuthota and Lava Bhargava. Transformer based intrusion detection for iot networks. IEEE Internet of Things Journal, 2025.
[28] Ibrahim A Fares, Ahmed G Abdellatif, Mohamed Abd Elaziz, Mansour Shrahili, Adham Elmahallawy, Rana Muhammad Sohaib, Mahmoud A Shawky, and Syed Tariq Shah. Deep transfer learning based on hybrid swin transformers with lstm for intrusion detection systems in iot environment. IEEE Open Journal of the Communications Society, 2025.
[29] C Karpagavalli and M Kaliappan. Edge implicit weighting with graph transformers for robust intrusion detection in internet of things network. Computers & Security, 150:104299, 2025.
[30] Bashar Hamad Aubaidan, Rabiah Abdul Kadir, and Mohamad Taha Ijab. A comparative analysis of smote and cssf techniques for diabetes classification using imbalanced data. J. Comput. Sci, 20:1146–1165, 2024.
[31] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ar. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017.
[32] Md Sazzad Hossain, John M Betts, and Andrew P Paplinski. Dual focal loss to address class imbalance in semantic segmentation. Neurocomputing, 462:69–87, 2021.
[33] Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. Efficient transformers: A survey. ACM Computing Surveys, 55(6):1–28, 2022.
[34] Ayesha S Dina, AB Siddique, and D Manivannan. A deep learning approach for intrusion detection in internet of things using focal loss function. Internet of Things, 22:100699, 2023.
[35] Hyeonseob Nam and Bohyung Han. Learning multi-domain convolutional neural networks for visual tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4293–4302, 2016.
[36] Laith Alzubaidi, Jinglan Zhang, Amjad J Humaidi, Ayad Al-Dujaili, Ye Duan, Omran Al-Shamma, Jos´e Santamar´ıa, Mohammed A Fadhel, Muthana Al-Amidie, and Laith Farhan. Review of deep learning: concepts, cnn architectures, challenges, applications, future directions. Journal of big Data, 8:1–74, 2021.
[37] Yulin Wang, Jinheng Wang, and Honglin Jin. Network intrusion detection method based on improved cnn in internet of things environment. Mobile Information Systems, 2022(1):3850582, 2022.
[38] Nickolaos Koroniotis, Nour Moustafa, Elena Sitnikova, and Benjamin Turnbull. Towards the development of realistic botnet dataset in the internet of things for network forensic analytics: Bot-iot dataset. Future Generation Computer
Systems, 100:779–796, 2019.
[39] Muhammad Shafiq, Zhihong Tian, Ali Kashif Bashir, Xiaojiang Du, and Mohsen Guizani. Corrauc: a malicious bot-iot traffic detection method in iot network using machine-learning techniques. IEEE Internet of Things Journal, 8(5):3242–3254, 2020.
[40] Nour Moustafa. A new distributed architecture for evaluating ai-based security systems at the edge: Network ton iot datasets. Sustainable Cities and Society, 72:102994, 2021.
[41] Tim M Booij, Irina Chiscop, Erik Meeuwissen, Nour Moustafa, and Frank TH Den Hartog. Ton iot: The role of heterogeneity and the need for standardization of features and attack types in iot network intrusion data sets. IEEE Internet of Things Journal, 9(1):485–496, 2021.
[42] Euclides Carlos Pinto Neto, Sajjad Dadkhah, Raphael Ferreira, Alireza Zohourian, Rongxing Lu, and Ali A Ghorbani. Ciciot2023: A real-time dataset and benchmark for large-scale attacks in iot environment. Sensors, 23(13):5941, 2023.
[43] M Zolanvari, M. A Teixeira, L Gupta, K. M. Khan, and R Jain. Wustl-iiot-2021 dataset for iiot cybersecurity research, Washington University in St Louis USA, October 2021. URL http://www.cse.wustl.edu/~jain/iiot2/index.html.