Volume 16 , Issue 2 , PP: 47-60, 2025 | Cite this article as | XML | Html | PDF | Full Length Article
R. Udaya Nirmala Mary 1 * , R. Santhosh 2 *
Doi: https://doi.org/10.54216/JCIM.160204
Multi-cloud computing is emerging as a transformative solution to meet the extensive computational demands of Internet of Things (IoT) devices. In networks with multiple devices and clouds, factors such as real-time computing requirements, fluctuating wireless channel conditions, and dynamic network scales introduce significant complexity. Addressing these challenges, along with the resource constraints of IoT devices, is essential for effective multi-cloud integration. This paper proposes a hybrid decision-offloading model that integrates continuous and discrete decision-making. IoT devices must learn to make coordinated decisions regarding cloud server selection, task offloading ratios, and local computation capacity. This dual-layer decision-making process involves managing both continuous and discrete variables, along with inter-device coordination, which poses considerable challenges. To address these, we introduce a probabilistic approach that transforms discrete actions, such as selecting a cloud server, into a continuous domain. We further develop a Privacy-Aware Multi-Agent Deep Reinforcement Learning (PA-MADRL) framework that combines centralized training with distributed execution. This framework minimizes overall system costs by considering energy consumption and cloud server rental fees. Each IoT device operates as an agent, autonomously learning efficient policies while alleviating its computational burden. Experimental results demonstrate that the PA-MADRL framework effectively adapts to dynamic network conditions, learning optimal offloading policies. It significantly outperforms four state-of-the-art deep reinforcement-learning models and two heuristic methods, achieving lower system costs and improved resource efficiency.
Task Offloading , Privacy , Deep Reinforcement Learning , Wireless Network , Internet of Things (IoT)
[1] P. Viktor and M. Fodor, “Examining Internet of Things (IoT) Devices: A Comprehensive Analysis,” in 2024 IEEE 22nd World Symposium on Applied Machine Intelligence and Informatics (SAMI), Jan. 2024, pp. 000115-000120.
[2] J. Chen, J. He, F. Chen, Z. Lv, J. Tang, W. Li, and G. Han, “Towards General Industrial Intelligence: A Survey on IIoT-Enhanced Continual Large Models,” arXiv preprint arXiv:2409.01207, 2024.
[3] F. Firouzi, B. Farahani, and A. Marinšek, “The convergence and interplay of edge, fog, and cloud in the AI-driven Internet of Things (IoT),” Information Systems, vol. 107, p. 101840, 2022.
[4] P. Popovski, F. Chiariotti, K. Huang, A. E. Kalør, M. Kountouris, N. Pappas, and B. Soret, “A perspective on time toward wireless 6G,” Proceedings of the IEEE, vol. 110, no. 8, pp. 1116-1146, 2022.
[5] K. Thakkar, A. Patel, S. Patel, K. Patel, A. Nayak, and A. Budhrani, “A Complete virtual Edge Computing Extrapolation Architectural style, Uses, and Implications,” in 2023 IEEE World Conference on Applied Intelligence and Computing (AIC), Jul. 2023, pp. 63-70.
[6] K. Wang, J. Jin, Y. Yang, T. Zhang, A. Nallanathan, C. Tellambura, and B. Jabbari, “Task offloading with multi-tier computing resources in next generation wireless networks,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 2, pp. 306-319, 2022.
[7] P. Zhang, N. Chen, G. Xu, N. Kumar, A. Barnawi, M. Guizani, and K. Yu, “Multi-target-aware dynamic resource scheduling for cloud-fog-edge multi-tier computing network,” IEEE Transactions on Intelligent Transportation Systems, 2023.
[8] M. Alabadi, A. Habbal, and X. Wei, “Industrial internet of things: Requirements, architecture, challenges, and future research directions,” Expert Systems, vol. 10, pp. 66374-66400, 2022.
[9] S. S. Gill, M. Xu, C. Ottaviani, P. Patros, R. Bahsoon, A. Shaghaghi, and S. Uhlig, “AI for next generation computing: Emerging trends and future directions,” Internet of Things, vol. 19, p. 100514, 2022.
[10] M. Yazdi, “Integration of IoT and Edge Computing in Industrial Systems,” in Advances in Computational Mathematics for Industrial System Reliability and Maintainability, Cham: Springer Nature Switzerland, 2024, pp. 121-137.
[11] G. Nain, K. K. Pattanaik, and G. K. Sharma, “Towards edge computing in intelligent manufacturing: Past, present and future,” Journal of Manufacturing Systems, vol. 62, pp. 588-611, 2022.
[12] R. Das and M. M. Inuwa, “A review on fog computing: issues, characteristics, challenges, and potential applications,” Telematics and Informatics Reports, vol. 10, p. 100049, 2023.
[13] M. Nikpour, P. B. Yousefi, H. Jafarzadeh, K. Danesh, R. Shomali, A. G. Lonbar, and M. Ahmadi, “Intelligent energy management with IoT framework in smart cities using intelligent analysis: An application of machine learning methods for complex networks and systems,” Networks, 2023.
[14] A. Hazra, P. Rana, M. Adhikari, and T. Amgoth, “Fog computing for next-generation internet of things: fundamental, state-of-the-art and research challenges,” Computer Science Review, vol. 48, p. 100549, 2023.
[15] D. Alsadie, “A Comprehensive Review of AI Techniques for Resource Management in Fog Computing: Trends, Challenges and Future Directions,” International Journal of Information Technology, 2024.
[16] W. Almuseelem, “Energy-efficient and security-aware task offloading for multi-tier edge-cloud computing systems,” Soft Computing, 2023.
[17] M. Hassan, A. A. Al-Awady, A. Ali, M. M. Iqbal, M. Akram, J. Khan, and A. A. AbuOdeh, “An efficient dynamic decision-based task optimization and scheduling approach for microservice-based cost management in mobile cloud computing applications,” Pervasive and Mobile Computing, vol. 92, p. 101785, 2023.
[18] A. B. Kanbar and K. Faraj, “Region aware dynamic task scheduling and resource virtualization for load balancing in IoT–fog multi-cloud environment,” Future Generation Computer Systems, vol. 137, pp. 70-86, 2022.
[19] N. Mungoli, “Scalable, Distributed AI Frameworks: Leveraging Cloud Computing for Enhanced Deep Learning Performance and Efficiency,” Sensors, 2023.
[20] G. K. Walia, M. Kumar, and S. S. Gill, “AI-empowered fog/edge resource management for IoT applications: A comprehensive review, research challenges and future perspectives,” IEEE Communications Surveys & Tutorials, 2023.
[21] H. Zhao, G. Lu, Y. Liu, Z. Chang, L. Wang, and T. Hämäläinen, “Safe DQN-based AOI-minimal task offloading for UAV-aided edge computing system,” IEEE Internet of Things Journal, 2024.
[22] C. Hou and Q. Zhao, “Optimal task-offloading control for edge computing system with tasks offloaded and computed in sequence,” IEEE Transactions on Automation Science and Engineering, vol. 20, no. 2, pp. 1378-1392, 2022.
[23] Mohanasundaram, “Graph Based Event Measurement for Analyzing Distributed Anomalies in Sensor Networks,” Sådhanå, vol. 45, p. 212, 2020.
[24] H. Jiang, X. Dai, Z. Xiao, and A. Iyengar, “Joint task offloading and resource allocation for energy-constrained mobile edge computing,” IEEE Transactions on Mobile Computing, vol. 22, no. 7, pp. 4000-4015, 2022.
[25] S. Long, Y. Zhang, Q. Deng, T. Pei, J. Ouyang, and Z. Xia, “An efficient task offloading approach based on multi-objective evolutionary algorithm in cloud-edge collaborative environment,” IEEE Transactions on Network Science and Engineering, vol. 10, no. 2, pp. 645-657, 2022.
[26] L. Zang, X. Zhang, and B. Guo, “Federated deep reinforcement learning for online task offloading and resource allocation in WPC-MEC networks,” Soft Computing, vol. 10, pp. 9856-9867, 2022.
[27] J. Almutairi, M. Aldossary, H. A. Alharbi, B. A. Yosuf, and J. M. Elmirghani, “Delay-optimal task offloading for UAV-enabled edge-cloud computing systems,” IEEE Access, vol. 10, pp. 51575-51586, 2022.
[28] C. Fang, X. Meng, Z. Hu, F. Xu, D. Zeng, M. Dong, and W. Ni, “AI-driven energy-efficient content task offloading in cloud-edge-end cooperation networks,” IEEE Open Journal of the Computer Society, vol. 3, pp. 162-171, 2022.
[29] T. H. Nguyen and L. Park, “A survey on deep reinforcement learning-driven task offloading in aerial access networks,” in 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), Oct. 2022, pp. 822-827.
[30] Y. Liu, Y. Mao, Z. Liu, F. Ye, and Y. Yang, “Joint task offloading and resource allocation in heterogeneous edge environments,” IEEE Transactions on Mobile Computing, 2023.
