Volume 16 , Issue 1 , PP: 208-230, 2025 | Cite this article as | XML | Html | PDF | Full Length Article
Huda Lafta Majeed 1 , Oday Ali Hassen 2 * , Dhyeauldeen Ahmed Farhan 3 , Yu Yu Gromov 4 , Kavita Sheoran 5 , Geetika Dhand 6
Doi: https://doi.org/10.54216/JCIM.160115
Smile detection and recognition have been a key component of sentiment analysis, social robotics, human-computer interaction, and mental health monitoring before the advent of deep learning. Understanding and accurately identifying smiles can provide deep insights into human behavior, strengthen communication systems, and enhance adaptive responses in AI interfaces. This paper is a comprehensive review of algorithms developed for smile detection and recognition, and categorizes their main approaches into three traditional computer vision techniques: feature-based, machine learning-based, and deep learning-based. These techniques rely on handcrafted features such as edges, geometric features of the face, and texture, which give interpretability and limited adaptability. This paper explores feature extraction methods such as geometric and histogram-based features (e.g., histograms of directed gradients). In addition, this paper evaluates the effectiveness of traditional classifiers, including support vector machines that use machine learning-based methods, leveraging algorithms such as support vector machines (SVMs), extracted features to classify smiles with improved accuracy. Deep learning techniques, especially convolutional neural networks (CNNs) and hybrid methods provide end-to-end learning capabilities, extracting features directly from raw pixel data and enabling real-time performance. These frameworks, including recurrent neural networks (RNNs) for temporal analysis, generative adversarial networks (GANs) for data augmentation, and graph neural networks (GNNs) for structural analysis, have also pushed the boundaries of smile detection in dynamic and challenging environments. It also aims to provide a comprehensive overview of these classical methods, and analyze their strengths, limitations, drawbacks, and performance across diverse datasets of the proposed databases by focusing on describing these datasets and researchers’ methods of working on them as benchmarks for their research, and highlighting their importance in the environments and their contributions to the development of smile detection algorithms in the field of computer vision. Among these datasets are datasets such as CK+, FER2013, AffectNet, and Jaffe in developing, training, and evaluating smile detection and recognition algorithm models. By comparing these methodologies, our paper recommends directing future research towards more efficient, robust, and scalable solutions for smile detection and recognition in diverse applications.
Smile detection algorithm , Facial expressions , Deep and machine learning of facial expressions , Computer vision real-time smile detection
[1] Saeed, V. A. (2024). A Framework for Recognition of Facial Expression Using HOG Features. International Journal of Mathematics, Statistics, and Computer Science, 2, 1–8. https://doi.org/10.59543/ijmscs.v2i.7815
[2] Huang, Z., Zheng, H., Li, C., & Che, C. (2024). Application of machine learning-based k-means clustering for financial fraud detection. Academic Journal of Science and Technology, 10(1), 33-39.
[3] Passos, L. A., Jodas, D., Costa, K. A., Souza Júnior, L. A., Rodrigues, D., Del Ser, J., ... & Papa, J. P. (2024). A review of deep learning‐based approaches for deepfake content detection. Expert Systems, 41(8), e13570.
[3] Sedaghatjoo, Z., Hosseinzadeh, H., & Bigham, B. S. (2024). Local Binary Pattern (LBP) Optimization for Feature Extraction. arXiv preprint arXiv:2407.18665.
[4] Aydın, Muhammed Taha, Oğuzhan Menemencioğlu, and İlhami Muharrem Orak. "FACE RECOGNITION APPROACH BY USING DLIB AND K-NN." Current Trends in Computing 1.2 (2024): 93-103.
[5] Fard, Ali Pourramezan, et al. "AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels." arXiv preprint arXiv:2410.22506 (2024).
[5] Rabea, Mahmoud, et al. "IdentiFace: A VGG based multimodal facial biometric system." arXiv preprint arXiv:2401.01227 (2024).
[6] Etemad, Shervin, et al. "SP25. Design And Implementation Of A Hybrid Paper-electronic Patient Data Tool For Operation Smile: Best Practices And Lessons Learned Across 20 Countries." Plastic and Reconstructive Surgery–Global Open 12.S4 (2024): 96-97.
[7] He, Ying, et al. "De novo Design of Biocompatible Nanomaterials Using Quasi-SMILES and Recurrent Neural Networks." ACS Applied Materials & Interfaces (2024).
[8] Rao, K. Venkateswara, Kunjam Nageswara Rao, and G. Sita Ratnam. "Accelerating Drug Safety Assessment using Bidirectional-LSTM for SMILES Data." arXiv preprint arXiv:2407.18919 (2024).
[9] Cherian, A. K., Vaidhehi, M., Arshey, M., Briskilal, J., & Simpson, S. V. (2024). Generative adversarial networks with stochastic gradient descent with momentum algorithm for video-based facial expression. International Journal of Information Technology, 16(6), 3703-3722.
[10] De Carlo, Alessandro, et al. "Predicting ADMET Properties from Molecule SMILE: A Bottom-Up Approach Using Attention-Based Graph Neural Networks." Pharmaceutics 16.6 (2024): 776.
[11] Costa, V. G., Pérez-Aracil, J., Salcedo-Sanz, S., & Pedreira, C. E. (2024). Evolving interpretable decision trees for reinforcement learning. Artificial Intelligence, 327, 104057.
[12] Yakar, Murat, et al. "An Innovative Fusion of New Smile Random Forest and Fuzzy Analysis for Comprehensive Seismic hazard Assessment: A Case Study in Mersin, Türkiye." (2024).
[13] Cherian, Aswathy K., et al. "A Robust Design of Real-Time Resilient Smile Recognition System using Hybrid Deep Learning Principles." 2024 10th International Conference on Communication and Signal Processing (ICCSP). IEEE, 2024.
[14] Vanamoju, Sri Vaibhav Mohan Dev, et al. "Facial Emotion Recognition using YOLO based Deep Learning Classifier." 2024 First International Conference on Electronics, Communication and Signal Processing (ICECSP). IEEE, 2024.
[15] Thomas, B., Bhatt, A., & Singh, S. N. (2024, August). Recognition of Facial Emotions Using CNN Architecture and FER2013. In 2024 International Conference on Electrical Electronics and Computing Technologies (ICEECT) (Vol. 1, pp. 1-6). IEEE.
[16] Fard, Ali Pourramezan, et al. "AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels." arXiv preprint arXiv:2410.22506 (2024).
[217] Abu, N. A., Abidin, Z. Z., & Darwish, S. M. (2021). A new descriptor for smile classification based on cascade classifier in unconstrained scenarios. Symmetry, 13(5), 805.
[18] Guijo-Rubio, David, et al. "Unsupervised feature-based algorithms for time series extrinsic regression." Data Mining and Knowledge Discovery (2024): 1-45.
[19] Antipona, Clarence A., et al. "An Enhancement of Haar Cascade Algorithm Applied to Face Recognition for Gate Pass Security." arXiv preprint arXiv:2411.03831 (2024).
[20] Sofronievski, Bojan, et al. "Efficient Real-time On-the-edge Facial Expression Recognition using Optomyography Smart Glasses." 2024 International Conference on Intelligent Environments (IE). IEEE, 2024.
[21] Akrom, Muhamad, Supriadi Rustad, and Hermawan Kresno Dipojono. "SMILES-based machine learning enables the prediction of corrosion inhibition capacity." MRS Communications (2024): 1-9.
[22] Patil, Nagabhushan. "An enhanced segmentation technique and improved support vector machine classifier for facial image recognition." International Journal of Intelligent Computing and Cybernetics 15.2 (2021): 302-317.
[23] Chen, Pai-Hsuen, Rong-En Fan, and Chih-Jen Lin. "A study on SMO-type decomposition methods for support vector machines." IEEE transactions on neural networks 17.4 (2024): 893-908.
[25] Mohammed, Hisham, Ben K. Daniel, and Mauro Farella. "Smile analysis in dentistry and orthodontics–a review." Journal of the Royal Society of New Zealand (2024): 1-14.
[26] Hassen. A., Abu, N. A.., & Darwish, S. M. (2022). Realistic Smile Expression Recognition Approach Using Ensemble Classifier with Enhanced Bagging. Computers, Materials & Continua, 70(2).
[27] Rajesh, T. R., Kuchipudi, L. N., Dammalapati, A., & Surendran, R. (2024, July). Attendance System Based on Facial Recognition Using OpenCV. In 2024 2nd International Conference on Sustainable Computing and Smart Systems (ICSCSS) (pp. 1529-1535).
[28] Pradeep, V., Sumukha, B. S., Richards, G. R., & Prashant, S. P. (2024, April). Facial Emotion Detection using CNN and OpenCV. In 2024 International Conference on Emerging Technologies in Computer Science for Interdisciplinary Applications (ICETCS) (pp. 1-6).
[29] Winarno, S., Nugraha, A., Muttaqin, A. N. I., Zarifa, Y., Salsabila, P. M., & Mumtaz, G. F. (2024). Comparison of ArcFace and Dlib Performance in Face Recognition with Detection Using YOLOv8. INOVTEK Polbeng-Seri Informatika, 9(2), 890-903.
[30] Dai, Y., Itai, T., Pei, G., Yan, F., Chu, Y., Jiang, X., ... & Zhao, Z. (2024). DeepFace: Deep learning-based framework to contextualize orofacial cleft-related variants during human embryonic craniofacial development. Human Genetics and Genomics Advances.
[31] Fard, A. P., Hosseini, M. M., Sweeny, T. D., & Mahoor, M. H. (2024). AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels. arXiv preprint arXiv:2410.22506.
[32] Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2017). Affectnet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, 10(1), 18-31.
[33] Cherian, A. K., Devipriya, S., Saoji, B. P., Mallikeswari, B., Thiagarajan, R., & Krishnamoorthy, R. (2024, April). A Robust Design of Real-Time Resilient Smile Recognition System using Hybrid Deep Learning Principles. In 2024 10th International Conference on Communication and Signal Processing (ICCSP) (pp. 592-596). IEEE.
[34] Saif, F. M., Rubayat, J., Pranto, F. S., Al Walid, M. H., & Ahammed, K. M. S. Facial Expressions Recognition Using Bezier Curve and Performance Analysis Using Ann, Naïve-Bayes and SVM Classifiers.
[35] Mattioli, M., & Cabitza, F. (2024). Not in My Face: Challenges and Ethical Considerations in Automatic Face Emotion Recognition Technology. Machine
