2692-4048ISSN (Online)
2770-0070ISSN (Print)
Volume 17 , Issue 1 , PP: 159-182, 2025 | Cite this article as | XML | Html | PDF | Full Length Article
Osamah A. Altammami 1 *
Doi: https://doi.org/10.54216/FPA.170112
This article focuses on improving the accuracy and efficiency of multimodal human motion analysis using advanced techniques. Initially, Generative Adversarial Networks (GANs) were used for skeletal enhancement, and then Contrast-Limited Adaptive Histogram Equalization (CLAHE) was applied on the enhanced images to check the quality Joint-level. Limb-level, Temporal, Statistical Features are effectively recovered from contrast enhancing images. Furthermore, with the selected optimal features acquired from PutterFish Customized Serval Optimizer (PFCSO), the RehabNet++ architecture that makes the human movement assessment has been trained. This PFCSO model has been developed based on the inspiration acquired from the Pufferfish Optimization Algorithm (POA) and the Serval Optimization algorithm (SOA), respectively. The RehabNet++ architecture includes an optimized Multilayer Perceptron (O-MLP), STR-ResNet architecture, Attention-based Convolutional Neural Networks and Transfer Learning. The O-MLP model has been formulated by optimizing the hidden layers of MLP using the PFCSO model. In addition, Grad-CAM visualization is included to provide a graphical description for model selection. A comparative study has been conducted to test the proposed deep learning algorithm against the original methods using the Kimore dataset. This analysis is implemented in PYTHON and is dedicated to multimodal human motion analysis.
Physical Rehabilitation , Generative Adversarial Network , PFCSO , O-MLP , RehabNet++
[1] Amatriain-Fernández, S., Murillo-Rodríguez, E.S., Gronwald, T., Machado, S. and Budde, H., 2020. Benefits of physical activity and physical exercise in the time of pandemic. Psychological Trauma: Theory, Research, Practice, and Policy, 12(S1), p.S264.
[2] Deb, S., Islam, M.F., Rahman, S. and Rahman, S., 2022. Graph convolutional networks for assessment of physical rehabilitation exercises. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30, pp.410-419.
[3] Yu, X. and Xiong, S., 2019. A dynamic time warping based algorithm to evaluate kinect-enabled home-based physical rehabilitation exercises for older people. Sensors, 19(13), p.2882.
[4] Zhang, W., Su, C. and He, C., 2020. Rehabilitation exercise recognition and evaluation based on smart sensors with deep learning framework. IEEE Access, 8, pp.77561-77571.
[5] Wu, N.N., Tian, H., Chen, P., Wang, D., Ren, J. and Zhang, Y., 2019. Physical exercise and selective autophagy: benefit and risk on cardiovascular health. Cells, 8(11), p.1436.
[6] Semwal, V., Singh, G., Crespo, U. and González, R., 2021. Heterogeneous Computing Model for Post-injury Walking Pattern restoration and Postural Stability Rehabilitation Exercise Recognition.
[7] Capecci, M., Ceravolo, M.G., Ferracuti, F., Iarlori, S., Monteriu, A., Romeo, L. and Verdini, F., 2019. The KIMORE dataset: KInematic assessment of MOvement and clinical scores for remote monitoring of physical REhabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 27(7), pp.1436-1448.
[8] Baltzer W I, Smithostrin S, Warnock J J, et al. Evaluation of the clinical effects of diet and physical rehabilitation in dogs following tibial plateau leveling osteotomy[J]. Journal of the American Veterinary Medical Association, 2018, 252(6):686-700.
[9] Rochester, C.L., 2019. Patient assessment and selection for pulmonary rehabilitation. Respirology, 24(9), pp.844-853.
[10] Jiang, Y., 2020. Combination of wearable sensors and internet of things and its application in sports rehabilitation. Computer Communications, 150, pp.167-176.
[11] Kitzman, D.W., Whellan, D.J., Duncan, P., Pastva, A.M., Mentz, R.J., Reeves, G.R., Nelson, M.B., Chen, H., Upadhya, B., Reed, S.D. and Espeland, M.A., 2021. Physical rehabilitation for older patients hospitalized for heart failure. New England Journal of Medicine, 385(3), pp.203-216.
[12] Fossati, C., Torre, G., Vasta, S., Giombini, A., Quaranta, F., Papalia, R. and Pigozzi, F., 2021. Physical exercise and mental health: The routes of a reciprocal relation. International Journal of Environmental Research and Public Health, 18(23), p.12364.
[13] Zhu, Z. A., Lu, Y. C., You, C. H., & Chiang, C. K. (2019). Deep learning forsensor-based rehabilitation exercise recognition and evaluation. Sensors, 19(4),887.
[14] Panwar, M., Biswas, D., Bajaj, H., Jöbges, M., Turk, R., Maharatna, K. and Acharyya, A., 2019. Rehab-net: Deep learning framework for arm movement classification using wearable sensors for stroke rehabilitation. IEEE Transactions on Biomedical Engineering, 66(11), pp.3026-3037.
[15] Sun, M., Zhao, S., Gilvary, C., Elemento, O., Zhou, J. and Wang, F., 2020. Graph convolutional networks for computational drug development and discovery. Briefings in bioinformatics, 21(3), pp.919-935.
[16] Liao, Y., Vakanski, A. and Xian, M., 2020. A deep learning framework for assessing physical rehabilitation exercises. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(2), pp.468-477.
[17] Esraa Mohamed, The Relationship between Artificial Intelligence and Internet of Things: A quick review, Journal of Cybersecurity and Information Management, Vol. 1 , No. 1 , (2020) : 30-34 (Doi : https://doi.org/10.54216/JCIM.010101)
[18] Capecci, M., Ceravolo, M.G., Ferracuti, F., Iarlori, S., Kyrki, V., Monteriu, A., Romeo, L. and Verdini, F., 2018. A Hidden Semi-Markov Model based approach for rehabilitation exercise assessment. Journal of biomedical informatics, 78, pp.1-11.
[19] Caru, M., Curnier, D., Bousquet, M. and Kern, L., 2020. Evolution of depression during rehabilitation program in patients with cardiovascular diseases. Disability and Rehabilitation, 42(3), pp.378-384.
[20] Mandolesi, L., Polverino, A., Montuori, S., Foti, F., Ferraioli, G., Sorrentino, P. and Sorrentino, G., 2018. Effects of physical exercise on cognitive functioning and wellbeing: biological and psychological benefits. Frontiers in psychology, p.509.
[21] Rahman, Z.U., Ullah, S.I., Salam, A., Rahman, T., Khan, I. and Niazi, B., 2022. Automated detection of rehabilitation exercise by stroke patients using 3-layer CNN-LSTM model. Journal of Healthcare Engineering, 2022.
[22] Schez-Sobrino, S., Vallejo, D., Monekosso, D.N., Glez-Morcillo, C. and Remagnino, P., 2020. A distributed gamified system based on automatic assessment of physical exercises to promote remote physical rehabilitation. IEEE Access, 8, pp.91424-91434.
[23] Iwao, Y., Shigeishi, H., Takahashi, S., Uchida, S., Kawano, S. and Sugiyama, M., 2019. Improvement of physical and oral function in community‐dwelling older people after a 3‐month long‐term care prevention program including physical exercise, oral health instruction, and nutritional guidance. Clinical and Experimental Dental Research, 5(6), pp.611-619.
[24] Whyte, J., Dijkers, M.P., Fasoli, S.E., Ferraro, M., Katz, L.W., Norton, S., Parent, E., Pinto, S.M., Sisto, S.A., Van Stan, J.H. and Wengerd, L., 2021. Recommendations for reporting on rehabilitation interventions. American Journal of Physical Medicine & Rehabilitation, 100(1), pp.5-16.
[25] Aditya Sharma , Aditya Vats , Shiv Shankar Dash , Surinder Kaur, Artificial Intelligence enabled virtual sixth sense application for the disabled, Fusion: Practice and Applications, Vol. 1 , No. 1 , (2020) : 32-39 (Doi : https://doi.org/10.54216/FPA.010104)
[26] Reem Atassi , Fuad Alhosban, Fusion Optimization and Classification Model for Blockchain Assisted Healthcare Environment, Fusion: Practice and Applications, Vol. 9 , No. 2 , (2022) : 62-73 (Doi : https://doi.org/10.54216/FPA.090205)
[27] Khder Alakkari, Alhumaima Ali Subhi, Hussein Alkattan, Ammar Kadi, Artem Malinin, Irina Potoroko, Mostafa Abotaleb, El-Sayed M El-kenawy, Forecasting COVID-19 Infection Using Encoder-Decoder LSTM and Attention LSTM Algorithms, Journal of Intelligent Systems and Internet of Things, Vol. 8 , No. 2 , (2023) : 20-33 (Doi : https://doi.org/10.54216/JISIoT.080202)
[28] Ashish Sharma , Yogesh Sharma , Radhika Bansal , Sushant Verma, Implementation of Crowd Sale using ERC-20 Tokens, Journal of Cybersecurity and Information Management, Vol. 2 , No. 1 , (2020) : 05-12 (Doi : https://doi.org/10.54216/JCIM.020101)
[29] Mustafa Altaee, Talib A., M. A. Jalil, Ali J., Thamer A. Alalwani, Intelligent Multi-Level Feature Fusion Using Remote Sensing and CNN Image Classification Algorithm, Journal of Intelligent Systems and Internet of Things, Vol. 9 , No. 1 , (2023) : 36-48 (Doi : https://doi.org/10.54216/JISIoT.090103)
