Real-Time Facial Emotion Detection Application with Image Processing Based on Convolutional Neural Network (CNN)

Ghaeril Juniawan Parel Hakim; Gandi Abetnego Simangunsong; Rangga Wasita Ningrat; Jonathan Cristiano Rabika; Muhammad Rafi' Rusafni; Endang Purnama Giri; Gema Parasti Mindara

doi:10.62951/ijeemcs.v1i4.123

Authors

Ghaeril Juniawan Parel Hakim Universitas IPB
Gandi Abetnego Simangunsong Universitas IPB
Rangga Wasita Ningrat Universitas IPB
Jonathan Cristiano Rabika Universitas IPB
Muhammad Rafi' Rusafni Universitas IPB
Endang Purnama Giri Universitas IPB
Gema Parasti Mindara Universitas IPB

DOI:

https://doi.org/10.62951/ijeemcs.v1i4.123

Keywords:

FER, Emotions, CNNs, OpenCV

Abstract

Facial Emotion Recognition (FER) is a key technology for identifying emotions based on facial expressions, with applications in human-computer interaction, mental health monitoring, and customer analysis. This study presents the development of a real-time emotion recognition system using Convolutional Neural Networks (CNNs) and OpenCV, addressing challenges such as varying lighting and facial occlusions. The system, trained on the FER2013 dataset, achieved 85% accuracy in emotion classification, demonstrating high performance in detecting happiness, sadness, and surprise. The results highlight the system's effectiveness in real-time applications, offering potential for use in mental health and customer behavior analysis.

References

Alshamsi H, Meng H, Li M. 2016. Real time facial expression recognition app development on mobile phones. Di dalam: 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD). [internet] 2016 12th International Conference on Natural Computation and 13th Fuzzy Systems and Knowledge Discovery (ICNC-FSKD);. Changsha, China. Changsha, China: IEEE. hlm. 1750–1755. [diunduh 2024 Nov 20]. Tersedia pada: http://ieeexplore.ieee.org/document/7603442/

Anzum F, Gavrilova ML. 2023. Emotion Detection From Micro-Blogs Using Novel Input Representation. IEEE Access. 11:19512–19522.doi:10.1109/ACCESS.2023.3248506.

Deshmukh S, Patwardhan M, Mahajan A. 2016. Survey on real‐time facial expression recognition techniques. IET Biom. 5(3):155–163.doi:10.1049/iet-bmt.2014.0104.

Hoang M-H, Kim S-H, Yang H-J, Lee G-S. 2021. Context-Aware Emotion Recognition Based on Visual Relationship Detection. IEEE Access. 9:90465–90474.doi:10.1109/ACCESS.2021.3091169.

Hossain MS, Muhammad G. 2019. Emotion recognition using deep learning approach from audio–visual emotional big data. Inf. Fusion. 49:69–78.doi:10.1016/j.inffus.2018.09.008.

Jaiswal A, Krishnama Raju A, Deb S. 2020. Facial Emotion Detection Using Deep Learning. Di dalam: 2020 International Conference for Emerging Technology (INCET). [internet] 2020 International Conference for Emerging Technology (INCET);. Belgaum, India. Belgaum, India: IEEE. hlm. 1–5. [diunduh 2024 Nov 20]. Tersedia pada: https://ieeexplore.ieee.org/document/9154121/

Khaireddin Y, Chen Z. Facial Emotion Recognition: State of the Art Performance on FER2013.

Khan R, Sharif O. 2017. A Literature Review on Emotion Recognition using Various Methods.

Ko B. 2018. A Brief Review of Facial Emotion Recognition Based on Visual Information. Sensors. 18(2):401.doi:10.3390/s18020401.

Ma Y, Hao Y, Chen M, Chen J, Lu P, Košir A. 2019. Audio-visual emotion fusion (AVEF): A deep efficient weighted approach. Inf. Fusion. 46:184–192.doi:10.1016/j.inffus.2018.06.003.

Mehendale N. 2020. Facial emotion recognition using convolutional neural networks (FERC). SN Appl. Sci. 2(3):446.doi:10.1007/s42452-020-2234-1.

Mellouk W, Handouzi W. 2020. Facial emotion recognition using deep learning: review and insights. Procedia Comput. Sci. 175:689–694.doi:10.1016/j.procs.2020.07.101.

Paiva-Silva AID, Pontes MK, Aguiar JSR, De Souza WC. 2016. How do we evaluate facial emotion recognition? Psychol. Neurosci. 9(2):153–175.doi:10.1037/pne0000047.

Sabri N. 2020. Student Emotion Estimation Based on Facial Application in E-Learning during COVID-19 Pandemic. Int. J. Adv. Trends Comput. Sci. Eng. 9(1.4):576–582.doi:10.30534/ijatcse/2020/8091.42020.

Schoneveld L, Othmani A, Abdelkawy H. 2021. Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition. Pattern Recognit. Lett. 146:1–7.doi:10.1016/j.patrec.2021.03.007.

Singh NT, Rana S, Kumari S, Ritu. 2023. Facial Emotion Detection Using Haar Cascade and CNN Algorithm. Di dalam: 2023 International Conference on Circuit Power and Computing Technologies (ICCPCT). [internet] 2023 International Conference on Circuit Power and Computing Technologies (ICCPCT);. Kollam, India. Kollam, India: IEEE. hlm. 931–935. [diunduh 2024 Nov 20]. Tersedia pada: https://ieeexplore.ieee.org/document/10245125/

Talegaonkar I, Joshi K, Valunj S, Kohok R, Kulkarni A. 2019. Real Time Facial Expression Recognition using Deep Learning. SSRN Electron. J..doi:10.2139/ssrn.3421486. [diunduh 2024 Nov 20]. Tersedia pada: https://www.ssrn.com/abstract=3421486

Yang J, Qian T, Zhang F, Khan SU. 2021. Real-Time Facial Expression Recognition Based on Edge Computing. IEEE Access. 9:76178–76190.doi:10.1109/ACCESS.2021.3082641.