536 506
Full Length Article
Fusion: Practice and Applications
Volume 10 , Issue 2, PP: 108-121 , 2023 | Cite this article as | XML | Html |PDF

Title

Multi-Level Fusion for Facial Expression Recognition in Human Behavior Identification

  Aqeel Hussein 1 * ,   Ibraheem H. M. 2 ,   Sarah Ali Abdulkareem 3 ,   Ryam Ali Zubaid 4 ,   Noor Thamer 5

1  Department of Medical device technology Engineering, Alfarahidi University, Baghdad, Iraq
    (aqeel.hussien@alfarahidiuc.edu.iq)

2  Computer Communications Engineering Department, Alrafidain University College, Baghdad, Iraq
    (ibraheem.hatem.elc@ruc.edu.iq)

3  Department of computer engineering techniques, Al-turath University College, Baghdad, Iraq
    (sarah.ali@turath.edu.iq)

4  Department of computer engineering techniques, Mazaya University college, Thi Qar, Iraq
    (ryamzubaid34@gmail.com)

5  Accounting Department, Al-Mustaqbal University College , 51001 Hillah, Babylon , Iraq
    (noorthamer2020@mustaqbal-college.edu.iq)

Full Text Download


Doi   :   https://doi.org/10.54216/FPA.100210

Received: November 23, 2022 Accepted: March 17, 2023

Abstract :

In this study, we present a multi-level fusion of deep learning technique for facial expression identification, with applications spanning the fields of cognitive science, personality development, and the detection and diagnosis of mental health disorders in humans. The suggested approach, named Deep Learning aided Hybridized Face Expression Recognition system (DLFERS), classifies human behavior from a single image frame through the use of feature extraction and a support vector machine. An information classification algorithm is incorporated into the methodology to generate a new fused image consisting of two integrated blocks of eyes and mouth, which are very sensitive to changes in human expression and relevant for interpreting emotional expressions. The Transformation of Invariant Structural Features (TISF) and the Transformation of Invariant Powerful Movement (TIPM) are utilized to extract features in the suggested method's Storage Pack of Features (SPOF). Multiple datasets are used to compare the effectiveness of different neural network algorithms for learning facial expressions. The study's major findings show that the suggested DLFERS approach achieves an overall classification accuracy of 93.96 percent and successfully displays a user's genuine emotions during common computer-based tasks.

Keywords :

Artificial Intelligence , Facial Expressions; Deep fusion features; Smart Decision System; Behavior traits.

References :

[1]  Elfenbein HA. Nonverbal dialects and accents in facial expressions of emotion. Emotion Review. 2013 Jan;5(1):90-6.

[2]  Weise T, Bouaziz S, Li H, Pauly M. Realtime performance-based facial animation. ACM transactions on graphics (TOG). 2011 Jul 25;30(4):1-0.

[3]  Bourke C, Douglas K, Porter R. Processing of facial emotion expression in major depression: a review. Australian and New Zealand Journal of Psychiatry. 2010 Aug 1;44(8):681-96.

[4]  Garg  A,  Bajaj  R.  Facial  expression  recognition  &  classification  using  hybridization  of  ica,  ga,  and neural network for human-computer interaction. Journal of Network Communications and Emerging Technologies (JNCET) www. jncet. org. 2015 May;2(1).

[5]  Olney  NT,  Goodkind  MS,  Lomen-Hoerth  C,  Whalen  PK,  Williamson  CA,  Holley  DE,  Verstaen  A, Brown  LM,  Miller  BL,  Kornak  J,  Levenson  RW.  Behaviour,  physiology  and  experience  of pathological laughing and crying in amyotrophic lateral sclerosis. Brain. 2011 Dec 1;134(12):3458-69.

[6]  Wood  A,  Rychlowska  M,  Korb  S,  Niedenthal  P.  Fashioning  the  face:  sensorimotor  simulation contributes to facial expression recognition. Trends in cognitive sciences. 2016 Mar 1;20(3):227 -40.

[7]  Shin E. A Neural Network approach to  Automated Essay Scoring: A Comparison with the Method of Integrating Deep Language Features using Coh-Metrix.

[8]  O'Connor  RC,  Nock  MK.  The  psychology  of  suicidal  behaviour.  The  Lancet  Psychiatry.  2014  Jun 1;1(1):73-85.

[9]  Wong N, Beidel DC, Sarver DE, Sims  V. Facial emotion recognition in children with high functioning autism  and  children  with  social  phobia.  Child  Psychiatry  &  Human  Development.  2012  Oct 1;43(5):775-94.

[10]  Kaplan A, Haenlein  M. Siri,  Siri, in my hand: Who's the  fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business Horizons. 2019 Jan 1;62(1):15 -25.

[11]  Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N. Localizing parts of  faces using a consensus of exemplars. IEEE transactions on pattern analysis and machine intelligence. 2013 Jan 15;35(12):2930 -40.

[12]  Galbally J, Marcel S, Fierrez J. Biometric antispoofing methods: A survey in face recognition. IEEE Access. 2014 Dec 18;2:1530-52.

[13]  Ranjan R, Castillo CD, Chellappa R. L2-constrained softmax loss for discriminative face verification. arXiv preprint arXiv:1703.09507. 2017 Mar 28.

[14]  Stins JF, Roelofs K, Villan J, Kooijman K, Hagenaars MA, Beek PJ. Walk to me when I smile,  step back  when  I'm  angry:  emotional  faces  modulate  whole-body  approach–avoidance  behaviors. Experimental brain research. 2011 Aug 1;212(4):603-11.

[15]  Giannopoulos P, Perikos I, Hatzilygeroudis I. Deep learning approaches for facial emotion recognition: A  case  study  on  fer-2013.  InAdvances  in  Hybridization  of  Intelligent  Methods  2018  (pp.  1-16). Springer, Cham.

[16]  Qi  C,  Li  M,  Wang  Q,  Zhang  H,  Xing  J,  Gao  Z,  Zhang  H.  Facial  expressions  recognition  based  on cognition and mapped binary patterns. IEEE Access. 2018 Mar 22;6:18795 -803.

[17]  Liu  P,  Han  S,  Meng  Z,  Tong  Y.  Facial  expression  recognition  via  a  boosted  deep  belief  network. InProceedings  of  the  IEEE  conference  on  computer  vision  and  pattern  r ecognition  2014  (pp.  1805-1812).

[18]  Dornaika  F,  Moujahid  A,  Raducanu  B.  Facial  expression  recognition  using  tracked  facial  actions: Classifier  performance  analysis.  Engineering  Applications  of  Artificial  Intelligence.  2013  Jan 1;26(1):467-77.

[19]  Boughrara H, Chtourou M, Amar CB, Chen L. Facial expression recognition based on a mlp neural network  using  constructive  training  algorithm.  Multimedia  Tools  and  Applications.  2016  Jan 1;75(2):709-31.

[20]  Khandait SP, Thool RC, Khandait PD. Automatic facial feature extraction and expression recognition based on neural network. arXiv preprint arXiv:1204.2073. 2012 Apr 10.

[21]  Boz H, Kose U. Emotion extraction from facial expressions by using artificial intelligence techniques. BRAIN. Broad Research in Artificial Intelligence and Neuroscience. 2018 Feb 28;9(1):5-16.

[22]  Zhang K, Huang Y, Du Y, Wang L. Facial expression recognition based on deep evolutional spatialtemporal networks. IEEE Transactions on Image Processing. 2017 Mar 30;26(9):4193 -203.

[23]  Perikos  I,  Ziakopoulos  E,  Hatzilygeroudis  I.  Recognizing  emotions  from  facial  expressions  using neural  network.  on  Artificial  Intelligence  Applications  and  Innovations  2014  Sep  19  (pp.  236-245). Springer, Berlin, Heidelberg.

[24]  Zhang  T,  Zheng  W,  Cui  Z,  Zong  Y,  Yan  J,  Yan  K.  A  deep  neural  network-driven  feature  learning method  for  multi-view  facial  expression  recognition.  IEEE  Transactions  on  Multimedia.  2016  Aug 3;18(12):2528-36.

[25]  Wen G, Hou Z, Li H, Li D, Jiang L, Xun E. Ensemble of deep neural networks with probability-based fusion for facial expression recognition. Cognitive Computation. 2017 Oct 1;9(5):597 -610.

[26]  Ali, M.H., Jaber, M.M., Abd, S.K., Alkhayyat, A. and Jasim, A.D., 2022. Artificial Neural NetworkBased  Medical  Diagnostics  and  Therapeutics. International  Journal  of  Pattern  Recognition  and 

Artificial Intelligence. 

Cite this Article as :
Style #
MLA Aqeel Hussein, Ibraheem H. M., Sarah Ali Abdulkareem, Ryam Ali Zubaid, Noor Thamer. "Multi-Level Fusion for Facial Expression Recognition in Human Behavior Identification." Fusion: Practice and Applications, Vol. 10, No. 2, 2023 ,PP. 108-121 (Doi   :  https://doi.org/10.54216/FPA.100210)
APA Aqeel Hussein, Ibraheem H. M., Sarah Ali Abdulkareem, Ryam Ali Zubaid, Noor Thamer. (2023). Multi-Level Fusion for Facial Expression Recognition in Human Behavior Identification. Journal of Fusion: Practice and Applications, 10 ( 2 ), 108-121 (Doi   :  https://doi.org/10.54216/FPA.100210)
Chicago Aqeel Hussein, Ibraheem H. M., Sarah Ali Abdulkareem, Ryam Ali Zubaid, Noor Thamer. "Multi-Level Fusion for Facial Expression Recognition in Human Behavior Identification." Journal of Fusion: Practice and Applications, 10 no. 2 (2023): 108-121 (Doi   :  https://doi.org/10.54216/FPA.100210)
Harvard Aqeel Hussein, Ibraheem H. M., Sarah Ali Abdulkareem, Ryam Ali Zubaid, Noor Thamer. (2023). Multi-Level Fusion for Facial Expression Recognition in Human Behavior Identification. Journal of Fusion: Practice and Applications, 10 ( 2 ), 108-121 (Doi   :  https://doi.org/10.54216/FPA.100210)
Vancouver Aqeel Hussein, Ibraheem H. M., Sarah Ali Abdulkareem, Ryam Ali Zubaid, Noor Thamer. Multi-Level Fusion for Facial Expression Recognition in Human Behavior Identification. Journal of Fusion: Practice and Applications, (2023); 10 ( 2 ): 108-121 (Doi   :  https://doi.org/10.54216/FPA.100210)
IEEE Aqeel Hussein, Ibraheem H. M., Sarah Ali Abdulkareem, Ryam Ali Zubaid, Noor Thamer, Multi-Level Fusion for Facial Expression Recognition in Human Behavior Identification, Journal of Fusion: Practice and Applications, Vol. 10 , No. 2 , (2023) : 108-121 (Doi   :  https://doi.org/10.54216/FPA.100210)
↑