Advanced Facial Emotion Recognition Using Machine Learning Models with Feature Extraction Techniques
Keywords:
Convolutional neural networks, Data augmentation, Deep learning, Facial Expression Recognition (FER), Transfer learningAbstract
This study presents a deep learning-based system for automatic Facial Expression Recognition (FER), targeting emotions such as happiness, sadness, and anger using the FER2013 dataset. A Convolutional Neural Network (CNN) forms the core of the model, enhanced with preprocessing steps like image resizing, normalization, and data augmentation to improve accuracy and generalization. Transfer learning with pre-trained models such as VGG16 and ResNet50 significantly boosts performance and reduces training time. The system is evaluated using accuracy, precision, recall, and F1-score, with hyper parameter tuning and early stopping employed to optimize results. The model achieves competitive accuracy, though challenges remain with subtle expressions and occlusions, pointing to future work involving attention mechanisms and multimodal inputs for improved robustness.
References
T. D. Le, L. T. Van, and Q. N. Hong, "Deep convolutional neural networks for emotion recognition of Vietnamese," International Journal of Machine Learning and Computing, vol. 10, no. 5, pp. 692–699, Sep. 2020, doi: https://doi.org/10.18178/ijmlc.2020.10.5.992
J. Li, S. Huang, X. Zhang, X. Fu, C. C. Chang, Z. Tang, and Z. Luo, "Facial expression recognition by transfer learning for small datasets," in Proceedings of the Second International Conference on Security with Intelligent Computing and Big Data Services (SICBS-2018), Springer, 2020, pp. 756–770, doi: https://doi.org/10.1007/978-3-030-16946-6_62
X. Wang, K. Wang, and S. Lian, "A survey on face data augmentation for the training of deep neural networks," Neural Computing and Applications, vol. 32, no. 19, pp. 15503–15531, Oct. 2020, doi: https://doi.org/10.1007/s00521-020-04748-3
S. Li and W. Deng, "Deep facial expression recognition: A survey," IEEE Transactions on Affective Computing, vol. 13, no. 3, pp. 1195–1215, Mar. 2020, doi: https://doi.org/10.1109/TAFFC.2020.2981446
M. Przybyła-Kasperek and K. Zieliński, "Comparative analysis of convolutional neural network architectures for emotion recognition in facial images," in Asian Conference on Intelligent Information and Database Systems, Apr. 17, 2025, pp. 245–261. doi: https://doi.org/10.1007/978-981-96-5887-9_18
X. Cai, J. Yuan, R. Zheng, L. Huang, and K. Church, "Speech emotion recognition with multi-task learning," in Proc. InInterspeech, vol. 2021, pp. 4508–4512, Aug. 2021, doi: https://doi.org/10.21437/Interspeech.2021-1852
Y. Wang, H. Ai, B. Wu, and C. Huang, "Real time facial expression recognition with adaboost," in Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), vol. 3, pp. 926–929, Aug. 2004, doi: https://doi.org/10.1109/ICPR.2004.1334680
P. B. Suresha, C. Hegde, Z. Jiang, and G. D. Clifford, "An edge computing and ambient data capture system for clinical and home environments," Sensors, vol. 22, no. 7, p. 2511, Mar. 2022, doi: https://doi.org/10.3390/s22072511
R. Kumar, G. Corvisieri, T. F. Fici, S. I. Hussain, D. Tegolo, and C. Valenti, "Transfer learning for facial expression recognition," Information, vol. 16, no. 4, p. 320, Apr. 2025, doi: https://doi.org/10.3390/info16040320
V. Suresh, G. Yeo, and D. C. Ong, "Critically examining the domain generalizability of facial expression recognition models," Arxiv Preprint Arxiv: 2106.15453, Jun. 29, 2021. doi: https://doi.org/10.48550/arXiv.2106.15453
