Volume 15 , Issue 1 , PP: 179-196, 2025 | Cite this article as | XML | Html | PDF | Full Length Article
Mohammed Qassim Matrood 1 * , Majid Hamid Ali 2
Doi: https://doi.org/10.54216/JCIM.150114
The issue of multi-access services based on the rapidly expanding Internet affects communication networks and creates congestion problems in buffers, which require effective control. Buffers have previously been managed using simple algorithms such as Droptail (DT), but this method has proven to have many setbacks, such as large queue delays and frequent occurrences of global synchronizations and shutdowns. To overcome these problems, the Active Queue Management (AQM) technique was introduced, including algorithms like Random Early Detection (RED). AQM techniques predict and discharge packets or label them before the buffer reaches its capacity to prevent congestion. In recent work, these algorithms have been enhanced with deep reinforcement learning to achieve improved network performance. This paper intends to present an evaluation of different studies conducted by researchers on congestion control methods. More importantly, it aims to compare the various findings, highlight the prospects of the different methods amid their weaknesses, and discuss future research opportunities within this critical domain of network management.
Random Early Detection , RED , Deep Reinforcement Learning , Machine learning , DRL
[1] Darch Abed Dawar, A. (2024). Enhancing Wireless Security and Privacy: A 2-Way Identity Authentication Method for 5G Networks. International Journal of Mathematics, Statistics, and Computer Science, 2, 183–198. https://doi.org/10.59543/ijmscs.v2i.9073
[2] A. B. Yousif, H. J. Hassan, and G. Muttasher, "Applying reinforcement learning for random early detaction algorithm in adaptive queue management systems," Indonesian Journal of Electrical Engineering and Computer Science, vol. 26, no. 3, pp. 1684-1691, 2022.
[3] A. B. Yousif, H. J. Hassan, and G. Muttasher, "Intelligent Parameter Tuning using Deep Q-network in Adaptive Queue Management Systems," IRAQI JOURNAL OF COMPUTERS, COMMUNICATIONS, CONTROL AND SYSTEMS ENGINEERING, vol. 22, no. 3, 2022.
[4] K. Nichols and V. Jacobson, "Controlling queue delay," Communications of the ACM, vol. 55, no. 7, pp. 42-50, 2012.
[5] S. Floyd and V. Jacobson, "Random early detection gateways for congestion avoidance," IEEE/ACM Transactions on networking, vol. 1, no. 4, pp. 397-413, 1993.
[6] R. Pan et al., "PIE: A lightweight control scheme to address the bufferbloat problem," in 2013 IEEE 14th international conference on high performance switching and routing (HPSR), 2013: IEEE, pp. 148-155.
[7] M. M. Hamdi, H. F. Mahdi, M. S. Abood, R. Q. Mohammed, A. D. Abbas, and A. H. Mohammed, "A review on queue management algorithms in large networks," in IOP Conference Series: Materials Science and Engineering, 2021, vol. 1076, no. 1: IOP Publishing, p. 012034.
[8] H. Fawaz, J. Lesca, P. T. A. Quang, J. Leguay, D. Zeghlache, and P. Medagliani, "Graph Convolutional Reinforcement Learning for Collaborative Queuing Agents," IEEE Transactions on Network and Service Management, 2022.
[9] S. Danladi and F. Ambursa, "„DyRED: An Enhanced Random Early Detection Based on a new Adaptive Congestion Control,“in proc. of 2019 15th International Conference on Electronics, Computer and Computation (ICECCO)," ed: Abuja, Nigeria, 2019.
[10] L. Alfat, D. Hanggoro, and R. F. Sari, "Performance evaluation of active queue management in fat tree architecture on data center network," in 2019 International Conference on Information and Communications Technology (ICOIACT), 2019: IEEE, pp. 22-27.
[11] A. F. AL-Allaf and A. A Jabbar, "RED with reconfigurable maximum dropping probability," International Journal of Computing and Digital Systems, vol. 8, no. 01, pp. 61-72, 2019.
[12] D. Marek, J. Szyguła, A. Domański, J. Domańska, K. Filus, and M. Szczygieł, "Adaptive Hurst-Sensitive active queue management," Entropy, vol. 24, no. 3, p. 418, 2022.
[13] A. Adamu, Y. Surajo, and M. T. Jafar, "SARED: Self-adaptive active queue management scheme for improving quality of service in network systems," Computer Science, vol. 22, 2021.
[14] A. A. Ismael and D. K. Türeli, "Study of a Smarter AQM Algorithm to Reduce Network Delay," 2022.
[15] T. Song and Y. Kyung, "Deep Reinforcement Learning Based Age-of-Information-Aware Low-Power Active Queue Management for IoT Sensor Networks," IEEE Internet of Things Journal, 2024.
[16] P. K. Donta, S. N. Srirama, T. Amgoth, and C. S. R. Annavarapu, "iCoCoA: Intelligent congestion control algorithm for CoAP using deep reinforcement learning," Journal of Ambient Intelligence and Humanized Computing, vol. 14, no. 3, pp. 2951-2966, 2023.
[17] P. Tam, S. Math, A. Lee, and S. Kim, "Multi-Agent Deep Q-Networks for Efficient Edge Federated Learning Communications in Software-Defined IoT," Computers, Materials & Continua, vol. 71, no. 2, 2022.
[18] E.-S. Jung and H. S. Kim, "RLECN—A learning based dynamic threshold control of ECN," ICT Express, vol. 9, no. 6, pp. 1007-1012, 2023.
[19] C. Pan, X. Cui, C. Zhao, Y. Wang, and Y. Wang, "An adaptive network congestion control strategy based on the change trend of average queue length," Computer Networks, p. 110566, 2024.
[20] L. C. de Almeida, W. R. D. da Silva, T. C. Tavares, R. Pasquini, C. Papagianni, and F. L. Verdi, "DESiRED--Dynamic, Enhanced, and Smart iRED: A P4-AQM with Deep Reinforcement Learning and In-band Network Telemetry," arXiv preprint arXiv:2310.18159, 2023.
[21] M. Kim, M. Jaseemuddin, and A. Anpalagan, "Deep reinforcement learning based active queue management for iot networks," Journal of Network and Systems Management, vol. 29, no. 3, p. 34, 2021.
[22] J. Bai, T. Zhang, and G. Xie, "MACC: Cross-Layer Multi-Agent Congestion Control with Deep Reinforcement Learning," arXiv preprint arXiv:2206.01972, 2022.
[23] M. Dery, O. Krupnik, and I. Keslassy, "QueuePilot: Reviving Small Buffers With a Learned AQM Policy," in IEEE INFOCOM 2023-IEEE Conference on Computer Communications, 2023: IEEE, pp. 1-10.
[24] A. Amanov, A. Majidi, N. Jahnabakhsh, and A. Çetin, "Adjusting ECN marking threshold in multi-queue DCNs with deep learning," The Journal of Supercomputing, vol. 79, no. 5, pp. 5443-5468, 2023.
[25] H. Fawaz, D. Zeghlache, T. A. Q. Pham, J. Leguay, and P. Medagliani, "Deep reinforcement learning for smart queue management," Electronic Communications of the EASST, vol. 80, 2021.
[26] K. Xiao, S. Mao, and J. K. Tugnait, "TCP-Drinc: Smart congestion control based on deep reinforcement learning," IEEE Access, vol. 7, pp. 11892-11904, 2019.
[27] Y. Lu, X. Ma, and Z. Xu, "Choose a correct marking position: ECN should be freed from tail mark," Computer Networks, vol. 197, p. 108329, 2021.
[28] A. Mirzaeinnia, M. Mirzaeinia, and A. Rezgui, "Latency and throughput optimization in modern networks: A comprehensive survey," arXiv preprint arXiv:2009.03715, 2020.
[29] J. Lorincz, Z. Klarin, and J. Ožegović, "A comprehensive overview of TCP congestion control in 5G networks: Research challenges and future perspectives," Sensors, vol. 21, no. 13, p. 4510, 2021.
[30] J. Liu and D. Wei, "Active Queue Management Based on Q-Learning Traffic Predictor," in 2022 International Conference on Cyber-Physical Social Intelligence (ICCSI), 2022: IEEE, pp. 399-404.
[31] M. H. Ali and S. Öztürk, "Efficient congestion control in communications using novel weighted ensemble deep reinforcement learning," Computers and Electrical Engineering, vol. 110, p. 108811, 2023.
[32] H. Ma, D. Xu, Y. Dai, and Q. Dong, "An intelligent scheme for congestion control: When active queue management meets deep reinforcement learning," Computer Networks, vol. 200, p. 108515, 2021.
