1
Department of Physics, College of Sciences, University of Bisha, PO Box 344, Bisha 61922, Saudi Arabia
(amabdelaty@ub.edu.sa)
2
Department of Pharmaceutics, College of Pharmacy, Qassim University, Al Qassim 51452, Saudi Arabia
(a.abdellatif@qu.edu.sa)
3
Department of Mathematics, College of Science and Humanities in Alkharj, Prince Sattam Bin Abdulaziz University, Alkharj 11942, Saudi Arabiaك School of Technology, Woxsen University- Hyderabad-502345, Telangana State, India.
(n.sooppy@psau.edu.sa)
4
School of Technology, Woxsen University- Hyderabad-502345, Telangana State, India
(shankar.rao@woxsen.edu.in)
5
Department of Computer Science, College of Science and Arts in Gurayat, Jouf University, Saudi Arabiaك Computer Science Department, Faculty of Computers and Information Assiut University, Egypt.
(rashamahmoud@aun.edu.eg)
6
Department of Computer Science, College of Science and Arts in Gurayat, Jouf University, Saudi Arabia; Information System Department, Faculty of Computers and Information, Assiut University, Assiut, Egypt.
(Taloba@aun.edu.eg)
Abstract :
Driver drowsiness has been identified as a major cause of roadway accidents globally. Efficiently determining the extent of drowsiness can greatly enhance preventive measures. This study proposes a novel approach, combining convolutional neural networks (CNN) and Gated Recurrent Units (GRU) to dynamically evaluate both the presence of drowsiness and its severity based on the Eye Aspect Ratio (EAR). By bridging spatial features extracted by CNNs with temporal sequences through GRU, our model offers a robust and real-time assessment of drowsiness levels, paving the way for enhanced safety measures in vehicular systems. Incorporating Neutrosophic Logic enables a more robust representation of uncertainty and ambiguity in the data and enhances the accuracy of driver drowsiness level detection within the Hybrid CNN-GRU framework. The model’s hybrid CNN-GRU structure combines CNN layers to extract spatial information from Human eye Images and GRU units to represent temporal correlations between frames. In-car cameras and sensors must be integrated to implement the suggested system in real-time and enable continuous driver behavior monitoring. The system alerts early warnings and takes action when drowsiness is detected, lowering the likelihood of accidents caused by weary drivers. The CNN-GRU hybrid architecture accurately detects fatigue during real-time driving. Performance metrics, including accuracy, recall, and F1-score, are provided for comparative research utilizing baseline models. Model behavior may be understood by visualizing tiredness detection and carefully examining false positives and negatives. The proposed CNN-GRU framework outperforms traditional methods such as SVM, KNN, and BPNN by achieving a significantly higher accuracy of 99.5%. It increases the recognition of driver tiredness by proposing a trustworthy and adaptable hybrid CNN-GRU deep learning system. This project is implemented in Python; it offers a practical and versatile solution for real-time driver drowsiness level detection. The proposed technology has the potential to dramatically increase traffic safety by sending out early warnings and taking steps to lessen the risks related to driver fatigue.
Keywords :
Driver drowsiness; Spatio-temporal analysis; Machine learning , Neural Networks; Gated Recurrent Unit; Neutrosophic Logic; Eye Aspect Ratio.
References :
[1] M. Dua, Shakshi, R. Singla, S. Raj, and A. Jangra, “Deep cnn models-based ensemble approach to driver drowsiness detection,” Neural Computing and Applications, vol. 33, pp. 3155–3168, 2021.
[2] B. Ganguly, D. Dey, and S. Munshi, “An integrated system for drivers’ drowsiness detection using deep learning frameworks,” in 2022 IEEE VLSI Device Circuit and System (VLSI DCS), pp. 55–59, IEEE, 2022.
[3] V. R. R. Chirra, S. R. Uyyala, and V. K. K. Kolli, “Deep cnn: A machine learning approach for driver drowsiness detection based on eye state.,” Rev. d’Intelligence Artif., vol. 33, no. 6, pp. 461–466, 2019.
[4] P. P. Patel, C. L. Pavesha, S. S. Sabat, and S. S. More, “Deep learning based driver drowsiness detection,” in 2022 International Conference on Applied Artificial Intelligence and Computing (ICAAIC), pp. 380– 386, IEEE, 2022.
[5] P. William, M. Shamim, A. R. Yeruva, D. Gangodkar, S. Vashisht, and A. Choudhury, “Deep learning based drowsiness detection and monitoring using behavioural approach,” in 2022 2nd International Conference on Technological Advancements in Computational Sciences (ICTACS), pp. 592–599, IEEE, 2022.
[6] M. F. Shakeel, N. A. Bajwa, A. M. Anwaar, A. Sohail, and A. Khan, “Detecting driver drowsiness in real time through deep learning based object detection,” in International work-conference on artificial neural networks, pp. 283–296, Springer, 2019.
[7] A. I. Taloba, “An artificial neural network mechanism for optimizing the water treatment process and desalination process,” Alexandria Engineering Journal, vol. 61, no. 12, pp. 9287–9295, 2022.
[8] K. Ravikumar, P. Chiranjeevi, N. M. Devarajan, C. Kaur, and A. I. Taloba, “Challenges in internet of things towards the security using deep learning techniques,” Measurement: Sensors, vol. 24, p. 100473, 2022.
[9] N. Muthukumaran, N. R. G. Prasath, and R. Kabilan, “Driver sleepiness detection using deep learning convolution neural network classifier,” in 2019 Third International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC), pp. 386–390, IEEE, 2019.
[10] A. Abozeid, R. Alanazi, A. Elhadad, A. I. Taloba, A. El-Aziz, M. Rasha, et al., “A large-scale dataset and deep learning model for detecting and counting olive trees in satellite imagery,” Computational Intelligence and Neuroscience, vol. 2022, 2022.
[11] F. Al-Sharqi and A. A.-Q. Abd Ghafur Ahmad, “Mapping on interval complex neutrosophic soft sets,” International Journal of Neutrosophic Science, vol. 19, no. 4, pp. 77–85, 2022.
[12] I. Silambarasan, R. Udhayakumar, F. Smarandache, and S. Broumi, “Some algebraic structures of neutrosophic fuzzy sets,” International Journal of Neutrosophic Science, vol. 19, no. 2, pp. 30–41, 2022.
[13] M. Leyva, P. Del Pozo, and A. Pe˜nafiel, “Neutrosophic dematel in the analysis of the causal factors of youth violence,” International Journal of Neutrosophic Science, vol. 18, no. 3, pp. 199–207, 2022.
[14] N. Omer, A. H. Samak, A. I. Taloba, and R. M. Abd El-Aziz, “A novel optimized probabilistic neural network approach for intrusion detection and categorization,” Alexandria Engineering Journal, vol. 72, pp. 351–361, 2023.
[15] M. Alqarni, A. H. Samak, S. S. Ismail, R. M. Abd El-Aziz, A. I. Taloba, et al., “Utilizing a neutrosophic fuzzy logic system with ann for short-term estimation of solar energy,” International Journal of Neutrosophic Science, vol. 20, no. 4, pp. 240–40, 2023.
[16] R. M. Abd El-Aziz, A. I. Taloba, and F. A. Alghamdi, “Quantum computing optimization technique for iot platform using modified deep residual approach,” Alexandria Engineering Journal, vol. 61, no. 12, pp. 12497–12509, 2022.
[17] L. Zhang, H. Saito, L. Yang, and J.Wu, “Privacy-preserving federated transfer learning for driver drowsiness detection,” IEEE Access, vol. 10, pp. 80565–80574, 2022.
[18] H. U. R. Siddiqui, A. A. Saleem, R. Brown, B. Bademci, E. Lee, F. Rustam, and S. Dudley, “Non-invasive driver drowsiness detection system,” Sensors, vol. 21, no. 14, p. 4833, 2021.
[19] E. Mag´an, M. P. Sesmero, J. M. Alonso-Weber, and A. Sanchis, “Driver drowsiness detection by applying deep learning techniques to sequences of images,” Applied Sciences, vol. 12, no. 3, p. 1145, 2022.
[20] W. Deng and R. Wu, “Real-time driver-drowsiness detection system using facial features,” Ieee Access, vol. 7, pp. 118727–118738, 2019.
[21] M. M. Abdelgwad, T. H. A. Soliman, and A. I. Taloba, “Arabic aspect sentiment polarity classification using bert,” Journal of Big Data, vol. 9, no. 1, pp. 1–15, 2022.
[22] S. M. Darwish, M. A. Salah, and A. A. Elzoghabi, “Identifying indoor objects using neutrosophic reasoning for mobility assisting visually impaired people,” Applied Sciences, vol. 13, no. 4, p. 2150, 2023.
[23] R. Abdubrani, M. Mustafa, and Z. L. Zahari, “A robust framework for driver fatigue detection from eeg signals using enhancement of modified z-score and multiple machine learning architectures,” IIUM Engineering Journal, vol. 24, no. 2, pp. 354–372, 2023.
[24] A. I. Taloba, A. A. Sewisy, and Y. A. Dawood, “Accuracy enhancement scaling factor of viola-jones using genetic algorithms,” in 2018 14th International Computer Engineering Conference (ICENCO), pp. 209–212, IEEE, 2018.
[25] “Drowsiness detection dataset.” https://www.kaggle.com/datasets/prasadvpatil/ mrl-dataset, Sept. 2023. Last accessed 18 Sep. 2023.
[26] S. Dey, S. A. Chowdhury, S. Sultana, M. A. Hossain, M. Dey, and S. K. Das, “Real time driver fatigue detection based on facial behaviour along with machine learning approaches,” in 2019 IEEE International Conference on Signal Processing, Information, Communication & Systems (SPICSCON), pp. 135–140, IEEE, 2019.
[27] T. Drutarovsky and A. Fogelton, “Eye blink detection using variance of motion vectors,” in European conference on computer vision, pp. 436–448, Springer, 2014.
[28] C. Dewi, R.-C. Chen, C.-W. Chang, S.-H. Wu, X. Jiang, and H. Yu, “Eye aspect ratio for real-time drowsiness detection to improve driver safety,” Electronics, vol. 11, no. 19, p. 3183, 2022.
| Style | # |
|---|---|
| MLA | Abdel-Haleem Abdel-Aty, Ahmed A. H. Abdellatif, Kottakkaran Sooppy Nisar, Shankar Rao Munjam, Rasha M. Abd El-Aziz, Ahmed I. Taloba. "Utilizing Neutrosophic Logic in a Hybrid CNN-GRU Framework for Driver Drowsiness Level Detection with Dynamic Spatio-Temporal Analysis Based on Eye Aspect Ratio." International Journal of Neutrosophic Science, Vol. 22, No. 2, 2023 ,PP. 144-161 (Doi : https://doi.org/10.54216/IJNS.220212) |
| APA | Abdel-Haleem Abdel-Aty, Ahmed A. H. Abdellatif, Kottakkaran Sooppy Nisar, Shankar Rao Munjam, Rasha M. Abd El-Aziz, Ahmed I. Taloba. (2023). Utilizing Neutrosophic Logic in a Hybrid CNN-GRU Framework for Driver Drowsiness Level Detection with Dynamic Spatio-Temporal Analysis Based on Eye Aspect Ratio. Journal of International Journal of Neutrosophic Science, 22 ( 2 ), 144-161 (Doi : https://doi.org/10.54216/IJNS.220212) |
| Chicago | Abdel-Haleem Abdel-Aty, Ahmed A. H. Abdellatif, Kottakkaran Sooppy Nisar, Shankar Rao Munjam, Rasha M. Abd El-Aziz, Ahmed I. Taloba. "Utilizing Neutrosophic Logic in a Hybrid CNN-GRU Framework for Driver Drowsiness Level Detection with Dynamic Spatio-Temporal Analysis Based on Eye Aspect Ratio." Journal of International Journal of Neutrosophic Science, 22 no. 2 (2023): 144-161 (Doi : https://doi.org/10.54216/IJNS.220212) |
| Harvard | Abdel-Haleem Abdel-Aty, Ahmed A. H. Abdellatif, Kottakkaran Sooppy Nisar, Shankar Rao Munjam, Rasha M. Abd El-Aziz, Ahmed I. Taloba. (2023). Utilizing Neutrosophic Logic in a Hybrid CNN-GRU Framework for Driver Drowsiness Level Detection with Dynamic Spatio-Temporal Analysis Based on Eye Aspect Ratio. Journal of International Journal of Neutrosophic Science, 22 ( 2 ), 144-161 (Doi : https://doi.org/10.54216/IJNS.220212) |
| Vancouver | Abdel-Haleem Abdel-Aty, Ahmed A. H. Abdellatif, Kottakkaran Sooppy Nisar, Shankar Rao Munjam, Rasha M. Abd El-Aziz, Ahmed I. Taloba. Utilizing Neutrosophic Logic in a Hybrid CNN-GRU Framework for Driver Drowsiness Level Detection with Dynamic Spatio-Temporal Analysis Based on Eye Aspect Ratio. Journal of International Journal of Neutrosophic Science, (2023); 22 ( 2 ): 144-161 (Doi : https://doi.org/10.54216/IJNS.220212) |
| IEEE | Abdel-Haleem Abdel-Aty, Ahmed A. H. Abdellatif, Kottakkaran Sooppy Nisar, Shankar Rao Munjam, Rasha M. Abd El-Aziz, Ahmed I. Taloba, Utilizing Neutrosophic Logic in a Hybrid CNN-GRU Framework for Driver Drowsiness Level Detection with Dynamic Spatio-Temporal Analysis Based on Eye Aspect Ratio, Journal of International Journal of Neutrosophic Science, Vol. 22 , No. 2 , (2023) : 144-161 (Doi : https://doi.org/10.54216/IJNS.220212) |