A Novel Reinforcement Learning-based Congestion Control Algorithm for DDoS-Induced Adversarial Conditions in Blockchain and Distributed Networks

Abedini, Ehsan; Jalaly Bidgoly, Amir; Nickray, Mohsen

doi:10.22042/isecure.2025.515662.1221

A Novel Reinforcement Learning-based Congestion Control Algorithm for DDoS-Induced Adversarial Conditions in Blockchain and Distributed Networks

Document Type : Research Article

Authors

Ehsan Abedini

Amir Jalaly Bidgoly

Mohsen Nickray

Department of Computer Engineering, University of Qom, Qom, Iran.

10.22042/isecure.2025.515662.1221

Abstract

Distributed Denial-of-Service (DDoS) attacks are among the most critical security threats to distributed network infrastructures, including blockchain systems. These attacks degrade performance, cause congestion, and disrupt service delivery or transaction processing. Traditional mitigation techniques have undergone extensive development. However, they often fail to intelligently detect and manage traffic patterns and struggle to adapt to dynamic conditions in decentralized environments. This paper proposes a reinforcement learning-based congestion control (CC) method that dynamically adjusts congestion window (CWND) following traditional TCP principles based on signals such as delay and packet loss. What distinguishes our approach is that the RL-agent interprets persistent or abnormal congestion patterns as potential indicators of adversarial high-load conditions (e.g., DDoS-induced congestion) and adapts CWND adjustments more intelligently to reduce their adverse. Leveraging the Q-learning algorithm, the proposed approach adapts dynamically to fluctuating traffic and conditions. Its learning capability enables continuous monitoring of behavior and timely responsiveness to anomalies, including sustained congestion patterns often associated with adversarial traffic surges. Simulation results across various DDoS scenarios—evaluated against conventional CC algorithms—demonstrate considerable improvements in key performance indicators such as reduced latency, enhanced bandwidth utilization, improved stability, decreased packet loss, and increased throughput. The proposed Q-learning-based CC operates at the peer-to-peer layer, regulating flow among blockchain nodes. It is independent of consensus mechanisms while indirectly improving consensus efficiency by reducing message delays and packet loss. This method offers a scalable and intelligent solution for cc under adversarial conditions, thereby contributing to improved robustness and efficiency in both general distributed systems and blockchain networks.

Keywords

Reinforcement Learning

Congestion Control

DDoS

Blockchain Security

Distributed Networks

Adversarial Conditions

[1] S. Nakamoto, Bitcoin: A peer-to-peer electronic cash system, 2008. [Online]. Available: https://bitcoin.org/bitcoin.pdf
[2] S. Singh et al., Blockchain Security Attacks, Challenges, and Solutions for the Future Distributed IoT Network, IEEE Access, vol. 9, pp. 13938–13959, 2021. [Online]. Available: https://doi.org/10.1109/ACCESS.2021.3051602
[3] C. D. Morar and D. E. Popescu, A Survey of Blockchain Applicability, Challenges, and Key Threats, Computers, vol. 13, no. 9, p. 223, Sept. 2024. [Online]. Available: https://doi.org/10.3390/computers13090223
[4] J. Liu and J. Wu, A Comprehensive Survey on Blockchain Technology and Its Applications, Highlights in Science, Engineering and Technology, vol. 85, pp. 128–138, Mar. 2024. [Online]. Available: https://doi.org/10.54097/r0ggyr24
[5] L. Wang et al., Security and Privacy Issues in Blockchain and Its Applications, IET Blockchain, vol. 3, no. 4, pp. 169–171, Dec. 2023. [Online]. Available: https://doi.org/10.1049/blc2.12051
[6] X. Li et al., Blockchain Security Threats and Collaborative Defense: A Literature Review, Computers, Materials & Continua, vol. 76, no. 3, pp. 2597–2629, 2023. [Online]. Available:
https://doi.org/10.32604/cmc.2023.040596
[7] R. Chaganti et al., A Survey on Blockchain Solutions in DDoS Attacks Mitigation: Techniques, Open Challenges and Future Directions, Computer Communications, vol. 197, pp. 96–112, Jan. 2023. [Online]. Available: https://doi.org/10.1016/j.comcom.2022.10.026
[8] M. Conti et al., A Survey on Security and Privacy Issues of Bitcoin, IEEE Communications Surveys & Tutorials, vol. 20, no. 4, pp. 3416–3452, 2018. [Online]. Available: https://doi.org/10.1109/COMST.2018.2842460
[9] R. Chaganti et al., A Comprehensive Review of Denial of Service Attacks in Blockchain Ecosystem and Open Challenges, IEEE Access, vol.10, pp. 96538–96555, 2022. [Online]. Available: https://doi.org/10.1109/ACCESS.2022.3205019

[10] P. Kamboj et al., Detection Techniques of DDoS Attacks: A Survey, in Proc. IEEE UPCON, 2017, pp. 675–679. [Online]. Available: https://doi.org/10.1109/UPCON.2017.8251130
[11] B. Ilyas et al., Prevention of DDoS Attacks Using an Optimized Deep Learning Approach in Blockchain Technology, Trans. Emerging Telecommunications Technologies, vol. 34, no. 4, p. e4729, Apr. 2023. [Online]. Available: https://doi.org/10.1002/ett.4729
[12] R. Jmal et al., Distributed Blockchain-SDN Secure IoT System Based on ANN to Mitigate DDoS Attacks, Applied Sciences, vol. 13, no. 8, p. 4953, Apr. 2023. [Online]. Available: https://doi.org/10.3390/app13084953
[13] A. A. Abdullah and S. A. Hussein, Detection and Mitigation Distribution Denial of Service Attack Based on Blockchain Concept, Ing´enierie Des Syst`emes d’Information, vol. 29, no. 3, pp. 1043–1049, Jun. 2024. [Online]. Available: https://doi.org/10.18280/isi.290322
[14] X. Li et al., An Adaptive DDoS Detection and Classification Method in Blockchain Using an Integrated Multi-Models, Computers, Materials & Continua, vol. 77, no. 3, pp. 3265–3288, 2023. [Online]. Available: https://doi.org/10.32604/cmc.2023.045588
[15] Kavitha and Ramalakshmi, Machine Learning-Based DDOS Attack Detection and Mitigation in SDNs for IoT Environments, Journal of the Franklin Institute, vol. 361, no. 17, p. 107197, Nov. 2024. [Online]. Available: https://doi.org/10.1016/j.jfranklin.2024.107197
[16] V. Hnamte et al., DDoS Attack Detection and Mitigation Using Deep Neural Network in SDN Environment, Computers & Security, vol. 138, p. 103661, Mar. 2024. [Online]. Available: https://doi.org/10.1016/j.cose.2023.103661
[17] Z. Shah et al., Blockchain Based Solutions to Mitigate Distributed Denial of Service (DDoS) Attacks in the Internet of Things (IoT): A Survey, Sensors, vol. 22, no. 3, p. 1094, Jan. 2022. [Online]. Available: https://doi.org/10.3390/s22031094
[18] S. Wani et al., Distributed Denial of Service (DDoS) Mitigation Using Blockchain—A Comprehensive Insight, Symmetry, vol. 13, no. 2, p. 227, Jan. 2021. [Online]. Available: https://doi.org/10.3390/sym13020227
[19] V. Jacobson, Congestion Avoidance and Control, ACM SIGCOMM Computer Communication Review, vol. 18, no. 4, pp. 314–329, Aug. 1988. [Online]. Available: https://doi.org/10.1145/52325.52356
[20] S. Ha et al., CUBIC: A New TCP-Friendly High-Speed TCP Variant, ACM SIGOPS Operating Systems Review, vol. 42, no. 5, pp. 64–74, Jul. 2008. [Online]. Available: https://doi.org/10.1145/1400097.1400105
[21] M. H. Ali and S. ¨ Ozt¨urk, Efficient Congestion Control in Communications Using Novel Weighted Ensemble Deep Reinforcement Learning, Computers and Electrical Engineering, vol. 110, p. 108811, Sep. 2023. [Online]. Available: https://doi.org/10.1016/j.compeleceng.2023.108811
[22] A. Kumar et al., Adaptive Contention Window Design Using Deep QLearning, arXiv, 2020. [Online]. Available: https://doi.org/10.48550/ARXIV.2011.09418
[23] H. K. Molia, Reinforcement Learning Based Adaptive Congestion Control for TCP over Wireless Networks, Int. J. of Computer Networks and Applications, vol. 11, no. 5, pp. 607–616, Oct. 2024. [Online]. Available: https://doi.org/10.22247/ijcna/2024/39
[24] J. Xing and M. Shahzad, A Reinforcement Learning Framework for Application-Specific TCP Congestion-Control, arXiv, 2025. [Online]. Available: https://doi.org/10.48550/ARXIV.2505.07042
[25] N. Gogoli et al., Shrew DDoS Attack Detection Based on Statistical Analysis, The ISC International Journal of Information Security, no. Online First, Jun. 2024. [Online]. Available:
https://doi.org/10.22042/isecure.2024.420803.1032
[26] B. Jang et al., Q-Learning Algorithms: A Comprehensive Classification and Applications, IEEE Access, vol. 7, pp. 133653–133667, 2019. [Online]. Available: https://doi.org/10.1109/ACCESS.2019.2941229
[27] S. Floyd and V. Jacobson, Random Early Detection Gateways for Congestion Avoidance, IEEE/ACM Trans. on Networking, vol. 1, no. 4, pp. 397–413, Aug. 1993. [Online]. Available: https://doi.org/10.1109/90.251892
[28] K. Fall and S. Floyd, Simulation-Based Comparisons of Tahoe, Reno and SACK TCP, ACM SIGCOMM Computer Communication Review, vol. 26, no. 3, pp. 5–21, Jul. 1996. [Online]. Available: https://doi.org/10.1145/235160.235162