Genetic Algorithms in Action: Adversarial Attacks on Machine Learning-Based XSS Detection Systems

Safi, Mohammadreza; Hadavi, Mohammad Ali

doi:10.22042/isecure.2026.248360

Genetic Algorithms in Action: Adversarial Attacks on Machine Learning-Based XSS Detection Systems

Document Type : Research Article

Authors

Mohammadreza Safi

Mohammad Ali Hadavi

Faculty of Electrical and Computer Engineering, Malek Ashtar University of Technology, Iran

10.22042/isecure.2026.248360

Abstract

Cross-Site Scripting (XSS) remains a critical web application vulnerability, consistently ranking among the OWASP Top 10 security risks. Although machine learning and deep learning techniques have improved XSS detection, these models are susceptible to adversarial attacks — carefully crafted inputs designed to evade detection. This paper proposes a novel adversarial attack framework that leverages a Genetic Algorithm to generate adversarial XSS payloads targeting machine learning-based detection systems automatically. Our framework is designed to achieve high transferability, enabling adversarial samples to bypass a wide range of detection models, even those with different architectures. By employing genetic operators such as selection, crossover, and mutation, the framework systematically optimizes payloads to maximize their ability to evade detection while preserving syntactic validity. Experimental results demonstrate that the generated adversarial samples consistently evade multiple state-of-the-art detection models, revealing significant vulnerabilities in current XSS defences. This work underscores the urgent need for more robust machine learning-based security solutions and provides a foundation for developing improved defences against adaptive adversarial threats.

Keywords

IMachine learning

Adversarial attack

Genetic Algorithms

XSS detection

Cross-Site Scripting

[1] OWASP Foundation. Cross Site Scripting (XSS), 2025. URL ttps://owasp.org/wwwcommunity/attacks/xss/. Accessed January 2025.

[2] OWASP Foundation. The OWASP Top Ten 2025, 2025. URL https://www.owasptopten. org/. Accessed January 2025.

[3] Qiuhua Wang, Hui Yang, Guohua Wu, KimKwang Raymond Choo, Zheng Zhang, Gongxun Miao, and Yizhi Ren. Black-box adversarial attacks on xss attack detection model. Computers & Security, 113:102554, 2022.

[4] Yong Fang, Cheng Huang, Yijia Xu, and Yang Li. Rlxss: Optimizing xss detection model to defend against adversarial attacks based on reinforcement learning. Future Internet, 11(8):177, 2019.

[5] Muralitharan Krishnan, Yongdo Lim, Seethalakshmi Perumal, and Gayathri Palanisamy. Detection and defending the xss attack using novel hybrid stacking ensemble learning-based dnn approach. Digital Communications and Networks, 10(3):716–727, 2024.

[6] Zhonglin Liu, Yong Fang, Cheng Huang, and Yijia Xu. Gaxss: effective payload generation method to detect xss vulnerabilities based on genetic algorithm. Security and Communication Networks, 2022(1):2031924, 2022.

[7] Charu Gupta, Rakesh Kumar Singh, and Amar Kumar Mohapatra. Geneminer: a classification approach for detection of xss attacks on web services. Computational Intelligence and Neuroscience, 2022(1):3675821, 2022.

[8] OWASP Foundation. XSS Filter Evasion Cheat Sheet, 2024. URL https://cheatsheetseries. owasp.org/cheatsheets/XSS_Filter_ Evasion_Cheat_Sheet.html. Accessed May 2024.

[9] GitHub. XSS Payload, Source URL, 2024. URL https://github.com/payloadbox/xsspayload-list/blob/master/Intruder/xsspayload-list.txt. Accessed May 2024.

[10] Kaggle. XSS Data, Source Classification, Online URL, 2024. URL https://www.kaggle.com/ code/hafizfarhad/cross-site-scriptingxss-prediction/input?select=XSS_dataset. csv. Accessed October 2024.