Static Malware Detection in Windows Executables Using Deep Neural Networks and Custom Binary Features

Rezaei, Sajjad; Fanian, Ali

doi:10.22042/isecure.2026.241272

Static Malware Detection in Windows Executables Using Deep Neural Networks and Custom Binary Features

Articles in Press

Document Type : Research Article

Authors

Sajjad Rezaei

Ali Fanian

Department of Computer Engineering, Isfahan University of Technology, Isfahan, Iran.

10.22042/isecure.2026.241272

Abstract

The extensive use of malware targeting Windows systems, particularly through Portable Executable (PE) files, has prompted significant research into malware detection. Although many approaches have been proposed, the increasing complexity and evasiveness of modern malware continue to present substantial challenges, underscoring the need for further advancements in detection strategies. This paper introduces a static malware detection framework based on deep learning and a set of carefully engineered binary features extracted directly from raw PE files. In contrast to conventional methods that rely on metadata or dynamic analysis, our approach performs detailed parsing of file headers, section layouts, entropy levels, import/export tables, and embedded resources to form a comprehensive feature set. A deep neural network is trained on these features, with its architecture and hyperparameters fine-tuned using Bayesian optimisation. The model is evaluated on a balanced dataset of benign and malicious PE files, achieving high accuracy (98.83%) and an F1-score of 98.95%. Fully automated and independent of dynamic execution or commercial tools, the proposed solution is well-suited for deployment in real-world applications such as antivirus systems and intrusion detection platforms.

Keywords

Binary Feature Extraction

Deep Neural Network

Malware Classification

Portable Executable Files

Static Analysis

[1] Mohammed Alshomrani, Aiiad Albeshri, Badraddin Alturki, Fouad Alallah, and Abdulaziz Alsulami. Survey of transformer-based malicious software detection systems. Electronics, 13(23):4677, 2024.

[2] Pradip Kunwar, Kshitiz Aryal, Maanak Gupta, Mahmoud Abdelsalam, and Elisa Bertino. Sok: Leveraging transformers for malware analysis. IEEE Transactions on Dependable and Secure Computing, 2024.

[3] Malik Ijaz Yousuf, Iftikhar Anwer, Muhammad Riasat, K.T. Zia, and Sung Kim. Windows malware detection based on static analysis with multiple features. PeerJ Computer Science, 9:e1319, 2023.

[4] Chunyu Miao, Liang Kou, Jilin Zhang, and Guozhong Dong. A lightweight malware detection model based on knowledge distillation. Mathematics, 12(24):4009, 2024.

[5] Pascal Maniriho, Abdun Naser Mahmood, and Mohammad Jabed Morshed Chowdhury. Earlymaldetect: A novel approach for early windows malware detection based on sequences of api calls. arXiv preprint, 2024.

[6] Ananya Redhu, Prince Choudhary, Kathiravan Srinivasan, and Tapan Kumar Das. Deep learning-powered malware detection in cyberspace: a contemporary review. Frontiers in Physics, 12:1349463, 2024.

[7] Daniel Gibert, Carles Mateu, and Jordi Planes. The rise of machine learning for detection and classification of malware: Research developments, trends and challenges. Journal of Network and Computer Applications, 153:102526, 2020.

[8] Ajit Kumar, K.S. Kuppusamy, and G. Aghila. A learning model to detect maliciousness of portable executable using integrated feature set. Journal of King Saud University - Computer and Information Sciences, 2017. URL http://dx. doi.org/10.1016/j.jksuci.2017.01.003.

[9] Tina Rezaei and Ali Hamze. An efficient approach for malware detection using pe header specifications. page 234, 2020.

[10] Prabhjot Singh et al. Feed-forward deep neural network (ffdnn)-based deep features for static malware detection. International Journal of Intelligent Systems, pages 1–20, 2023.

[11] Robertas Damaˇseviˇcius, Algirdas Venˇckauskas, Jevgenijus Toldinas, and ˇSaru¯nas Grigaliu¯nas. Ensemble-based classification using neural networks and machine learning models for windows pe malware detection. Electronics, 10(4):485, 2021.

[12] Daniel Gibert, Carles Mateu, and Jordi Planes. A survey of malware detection using deep learning. Machine Learning and Cybersecurity, 5:100227, 2024.

[13] Ananya Redhu et al. A comprehensive survey on deep learning approaches for malware detection: Taxonomy, current challenges, and future directions. IEEE Access, 12:45123–45145, 2024.

[14] Xingyuan Wei, Ce Li, Qiujian Lv, Ning Li, Degang Sun, and Yan Wang. Mitigating the impact of malware evolution on api sequence-based windows malware detector. arXiv preprint, 2024.

[15] Abdulwahab Ali Almazroi and Naveed Ayub. Deep learning hybridization for improved malware detection in smart internet of things. Scientific Reports, 14:7838, 2024.

[16] Carlos Contreras, Robert Baker, Arturo Guti´errez, and Jose Cerda. Static malware analysis using low-parameter machine learning models. Information, 13(3):59, 2024.

[17] Christofer Fellicious, Manuel Bischof, Kevin Mayer, Dorian Eikenberg, Stefan Hausotte, Hans P. Reiser, and Michael Granitzer. Malware detection based on api calls. arXiv preprint, 2025.

[18] Raj Kumar et al. Machine learning algorithm for malware detection: Taxonomy, current challenges, and future directions. IEEE Access, 11: 23456–23478, 2024.

[19] Spark Tsao. Faster and more accurate malware detection through predictive machine learning: Correlating static and behavioral features. Trend Micro Security Research, 2019. URL https://www.trendmicro. com/vinfo/us/security/news/securitytechnology/faster-and-more-accuratemalware-detection-through-predictivemachine-learning-correlating-staticand-behavioral features. Available at: https://www.trendmicro.com/vinfo/us/ security/news/security-technology/.

[20] Hyunghun Cho, Yongjin Kim, Eunjung Lee, Daeyoung Choi, Yongjae Lee, and Wonjong Rhee. Basic enhancement strategies when using bayesian optimization for hyperparameter tuning of deep neural networks. IEEE Access, 8: 52588–52608, 2020. . URL https://doi.org/ 10.1109/ACCESS.2020.2981072.

[21] Michael Lester. Pe malware machine learning dataset. Online dataset, 2021. URL https://practicalsecurityanalytics. com/pe-malware-machine-learningdataset/. Available at: https:// practicalsecurityanalytics.com/pemalware-machine-learning-dataset/.