287 255
Full Length Article
Journal of Intelligent Systems and Internet of Things
Volume 10 , Issue 2, PP: 102-112 , 2023 | Cite this article as | XML | Html |PDF

Title

Real Time E-learning Students Monitoring for Optimization Facial Landmark Recognition Based on Hybrid Deep Learning Techniques

  Shahad salh Ali 1 * ,   Jamila Harbi Al’Ameri 2 ,   Thekra Abbas 3

1  College of Science / Computer Science Department, Mustansiriyah University, Baghdad, Iraq
    (shahad.salh@uomustansiriyah.edu.iq)

2  College of Science / Computer Science Department, Mustansiriyah University, Baghdad, Iraq
    (dr.jameelahharbi@gmail.com)

3  College of Science / Computer Science Department, Mustansiriyah University, Baghdad, Iraq
    (thekra.abbas@uomustansiriyah.edu.iq)

Full Text Download


Doi   :   https://doi.org/10.54216/JISIoT.100209

Received: April 27, 2023 Revised: July 22, 2023 Accepted: October 17, 2023

Abstract :

The onset of digital education, propelled by the global COVID-19 crisis, has revolutionized the education sector, presenting unique difficulties, including the crucial task of preserving academic honesty. This study explores the possibilities of computer vision technologies, specifically face recognition and detection systems, to deter dishonest practices in online learning contexts. In this article we aim to construct efficacious strategies that leverage these technologies to track student actions in real-time and alert educators about possible cheating instances. This study presents two innovative models addressing cheating in online learning settings using cutting-edge computer vision techniques. Our initial model is an ensemble learning based face recognition system that blends the functionalities of three different deep learning (DL) structures: VGG, MobileNet, and DenseNet. This ensemble learning approach aims to offset the shortcomings of individual models while amplifying the overall effectiveness. The model’s efficiency will be gauged by juxtaposing it with other models and testing its performance against renowned benchmark datasets. Following this, we propose a second model designed for real-time face and cheating detection. This model integrates the FaceMesh model, facial landmarks analysis, and head pose estimation to identify possible cheating behaviors, such as significant shifts from a neutral or forward-facing head position. This model’s efficiency will be assessed through testing in simulated cheating scenarios and using authentic data from online learning contexts. Upon testing and validation, our proposed models have shown encouraging outcomes. The ensemble learning model outstripped individual models by attaining a remarkable accuracy rate of 91% through soft voting. Furthermore, the face detection system showcased sturdy abilities in recognizing faces under diverse conditions and accurately pinpointed potential cheating behaviors based on head pose estimation.

Keywords :

Face Identification; Face Detection; E-learning; Ensemble Learning; Deep Learning.

References :

[1]    Arif, Z. H., & Cengiz, K. (2023). Severity Classification for COVID-19 Infections based on Lasso-Logistic Regression Model. International Journal of Mathematics, Statistics, and Computer Science, 1, 25–32. https://doi.org/10.59543/ijmscs.v1i.7715

[2]    Colin Conrad, Qi Deng, Isabelle Caron, Oksana Shkurska, Paulette Skerrett, and Binod Sundararajan. How student perceptions about online learning difficulty influenced their satisfaction during canada’s covid-19 response. British Journal of Educational Technology, 53(3):534–557, 2022.

[3]    Simone CO Conceição. Faculty lived experiences in the online environment. Adult education quarterly, 57(1):26–45, 2006.

[4]    Margaret J Cox. Formal to informal learning with it: research challenges and issues for e-learning. Journal of computer assisted learning, 29(1):85–105, 2013.

[5]    AL Rohith Reddy, K Anish Gandhi, S Thakrimul Jaffer, and MK Srilekha. Student live behaviour monitoring during virtual class using artificial intelligence. In Journal of Physics: Conference Series, volume 2335, page 012026. IOP Publishing, 2022.

[6]    Hao Yang, Li Liu, Weidong Min, Xiaosong Yang, and Xin Xiong. Driver yawning detection based on subtle facial action recognition. IEEE Transactions on Multimedia, 23:572–583, 2020.

[7]    Anil Audumbar Pise, Hima Vadapalli, and Ian Sanders. Estimation of learning affects experienced by learners: an approach using relational reasoning and adaptive mapping. Wireless Communications and Mobile Computing, 2022, 2022.

[8]    Prabin Sharma, Shubham Joshi, Subash Gautam, Sneha Maharjan, Salik Ram Khanal, Manuel Cabral Reis, João Barroso, and Vítor Manuel de Jesus Filipe. Student engagement detection using emotion analysis, eye tracking and head movement with machine learning. In Technology and Innovation in Learning, Teaching and Education: Third International Conference, TECH-EDU 2022, Lisbon, Portugal, August 31–September 2, 2022, Revised Selected Papers, pages 52–68. Springer, 2023.

[9]    Omid Mohamad Nezami, Mark Dras, Len Hamey, Deborah Richards, Stephen Wan, and Cécile Paris. Automatic recognition of student engagement using deep learning and facial expression. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 273–289. Springer, 2020.

[10] CA Vázquez Rodríguez, M Mejía Lavalle, and R Pinto Elías. Modeling student engagement by means of nonverbal behavior and decision trees. In 2015 International Conference on Mechatronics, Electronics and Automotive Engineering (ICMEAE), pages 81–85. IEEE, 2015.

[11] Chinchu Thomas and Dinesh Babu Jayagopi. Predicting student engagement in classrooms using facial behavioral cues. In Proceedings of the 1st ACM SIGCHI international workshop on multimodal interaction for education, pages 33–40, 2017.

[12] Brais Martinez, Michel F Valstar, Bihan Jiang, and Maja Pantic. Automatic analysis of facial actions: A survey. IEEE transactions on affective computing, 10(3):325–347, 2017.

[13] Si Si Sann, Swe Swe Win, and Zin Minn Thant. An analysis of various image pre-processing techniques in butterfly image. International Journal for Advance Research and Development, 6(1):1–4, 2021.

[14] John T Hancock and Taghi M Khoshgoftaar. Survey on categorical data for neural networks. Journal of Big Data, 7(1):1–41, 2020.

[15] Anamika Dhillon and Gyanendra K Verma. Convolutional neural network: a review of models, methodologies and applications to object detection. Progress in Artificial Intelligence, 9(2):85–112, 2020.

[16] Miao Tian, Kai Ma, Zhihao Liu, Qinjun Qiu, Yongjian Tan, and Zhong Xie. Recognition of geological legends on a geological profile via an improved deep learning method with augmented data using transfer learning strategies. Ore Geology Reviews, page 105270, 2022.

[17] Andrey V Savchenko, Lyudmila V Savchenko, and Ilya Makarov. Classifying emotions and engagement in online learning based on a single facial expression recognition neural network. IEEE Transactions on Affective Computing, 13(4):2132–2143, 2022.

[18] Yi Zhu and Shawn Newsam. Densenet for dense flow. In 2017 IEEE international conference on image processing (ICIP), pages 790–794. IEEE, 2017.

[19] Eric Bauer and Ron Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine learning, 36:105–139, 1999.

[20] Sourya Dipta Das, Ayan Basak, and Saikat Dutta. A heuristic-driven ensemble framework for covid-19 fake news detection. In Combating Online Hostile Posts in Regional Languages during Emergency Situation: First International Workshop, CONSTRAINT 2021, Collocated with AAAI 2021, Virtual Event, February 8, 2021, Revised Selected Papers 1, pages 164–176. Springer, 2021.

[21] Richard Szeliski. Video mosaics for virtual environments. IEEE computer Graphics and Applications, 16(2):22–30, 1996.

[22] Byoung Chul Ko. A brief review of facial emotion recognition based on visual information. sensors, 18(2):401, 2018.

[23] Michal Podpora, Grzegorz Pawel Korbas, and Aleksandra Kawala-Janik. Yuv vs rgb-choosing a color space for human-machine interaction. In FedCSIS (Position Papers), pages 29–34. Citeseer, 2014.

[24] Yassine Hammadi, François Grondin, François Ferland, and Karina Lebel. Evaluation of various state of the art head pose estimation algorithms for clinical scenarios. Sensors, 22(18):6850, 2022.

[25] Nannan Wang, Xinbo Gao, Dacheng Tao, Heng Yang, and Xuelong Li. Facial feature point detection: A comprehensive survey. Neurocomputing, 275:50–65, 2018.

[26] Nataniel Ruiz, Eunji Chong, and James M Rehg. Fine-grained head pose estimation without keypoints. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 2074–2083, 2018.

[27] Ambi Singh and Smita Das. A cheating detection system in online examinations based on the analysis of eye-gaze and head-pose. In THEETAS 2022: Proceedings of The International Conference on Emerging Trends in Artificial Intelligence and Smart Systems, THEETAS 2022, 16-17 April 2022, Jabalpur, India, page 55. European Alliance for Innovation, 2022.

[28] Qiang Ji and Xiaojie Yang. Real-time eye, gaze, and face pose tracking for monitoring driver vigilance. Real-time imaging, 8(5):357–377, 2002.

Cite this Article as :
Style #
MLA Shahad salh Ali , Jamila Harbi Al’Ameri , Thekra Abbas. "Real Time E-learning Students Monitoring for Optimization Facial Landmark Recognition Based on Hybrid Deep Learning Techniques." Journal of Intelligent Systems and Internet of Things, Vol. 10, No. 2, 2023 ,PP. 102-112 (Doi   :  https://doi.org/10.54216/JISIoT.100209)
APA Shahad salh Ali , Jamila Harbi Al’Ameri , Thekra Abbas. (2023). Real Time E-learning Students Monitoring for Optimization Facial Landmark Recognition Based on Hybrid Deep Learning Techniques. Journal of Journal of Intelligent Systems and Internet of Things, 10 ( 2 ), 102-112 (Doi   :  https://doi.org/10.54216/JISIoT.100209)
Chicago Shahad salh Ali , Jamila Harbi Al’Ameri , Thekra Abbas. "Real Time E-learning Students Monitoring for Optimization Facial Landmark Recognition Based on Hybrid Deep Learning Techniques." Journal of Journal of Intelligent Systems and Internet of Things, 10 no. 2 (2023): 102-112 (Doi   :  https://doi.org/10.54216/JISIoT.100209)
Harvard Shahad salh Ali , Jamila Harbi Al’Ameri , Thekra Abbas. (2023). Real Time E-learning Students Monitoring for Optimization Facial Landmark Recognition Based on Hybrid Deep Learning Techniques. Journal of Journal of Intelligent Systems and Internet of Things, 10 ( 2 ), 102-112 (Doi   :  https://doi.org/10.54216/JISIoT.100209)
Vancouver Shahad salh Ali , Jamila Harbi Al’Ameri , Thekra Abbas. Real Time E-learning Students Monitoring for Optimization Facial Landmark Recognition Based on Hybrid Deep Learning Techniques. Journal of Journal of Intelligent Systems and Internet of Things, (2023); 10 ( 2 ): 102-112 (Doi   :  https://doi.org/10.54216/JISIoT.100209)
IEEE Shahad salh Ali, Jamila Harbi Al’Ameri, Thekra Abbas, Real Time E-learning Students Monitoring for Optimization Facial Landmark Recognition Based on Hybrid Deep Learning Techniques, Journal of Journal of Intelligent Systems and Internet of Things, Vol. 10 , No. 2 , (2023) : 102-112 (Doi   :  https://doi.org/10.54216/JISIoT.100209)
↑