Loading article...
Loading article...
Stream/Area:
-
Subject:
-
Paper ID:
IJSREM66493
Volume:
10 |
8 |
Aug 2026
Pages:
1-5
Publication Frequency:
Monthly (12 issues per year)
License:International Journal of Scientific Research in Engineering and Management follows the Creative Commons Attribution 4.0 (CC BY 4.0) License, allowing free use and distribution with proper attribution.
Department of Computer Science
University College of Engineering , Science & Technology Hyderabad (UCESTH) - JNTUH
Department of Computer Science
University College of Engineering , Science & Technology Hyderabad (UCESTH) - JNTUH
Software that can recognize how a person is feeling is becoming genuinely useful, and in this paper we approach that problem using two signals most systems already have access to: a photograph of a face and a short voice recording. Rather than train one combined model, we treated the two as separate problems - a 2-D CNN learns from the Balanced RAF-DB set of facial images, while a Conv1D network learns from MFCC features extracted from the RAVDESS speech dataset. Each model sorts its input into one of seven emotions: Angry, Disgust, Fear, Happy, Neutral, Sad or Surprise. Once trained, the two models were brought together behind a lightweight Streamlit interface, so a user only needs to upload an image or an audio clip to receive a predicted emotion along with a confidence score. When evaluated on data neither model had seen during training, both reached close to 96% accuracy, a result we found encouraging given how different the two input types are. Because the resulting pipeline is compact enough to run on ordinary hardware, it could reasonably be built into larger applications - patient monitoring, online learning platforms, or any interface where it helps to have a sense of the user's emotional state.
KEYWORDS
This paper set out to build and evaluate a system that reads human emotion from two independent channels - a face image and a voice recording - using a CNN for the former and an MFCC-driven Conv1D network for the latter, both presented through a single Streamlit interface. Trained on the Balanced RAF-DB and RAVDESS datasets, the two models each reached approximately 96% accuracy on test data, a result that supports the broader claim that a deep-learning pipeline of this kind can be accurate, lightweight, and usable by someone without a machine-learning background, with reasonably clear applications in healthcare, education, and other settings where it is useful to know something about a user's emotional state.
1. P. Ekman and W. V. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, 1978. 2. A. Mollahosseini, D. Chan and M. H. Mahoor, "Going deeper in facial expression recognition using deep neural networks," Proc. IEEE Winter Conf. on Applications of Computer Vision (WACV), 2016. 3. I. J. Goodfellow, Y. Bengio and A. Courville, Deep Learning. MIT Press, Cambridge, MA, USA, 2016. 4. Y. LeCun, Y. Bengio and G. Hinton, "Deep Learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015. 5. S. R. Livingstone and F. A. Russo, "The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS)," PLoS ONE, vol. 13, no. 5, 2018. 6. S. Li and W. Deng, "Reliable Crowdsourcing and Deep Locality-Preserving Learning for Facial Expression Recognition in the Wild," Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. 7. D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," Int. Conf. on Learning Representations (ICLR), 2015. 8. F. Pedregosa et al., "Scikit-learn: Machine Learning in Python," Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011. 9. A. Geron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 3rd ed., O'Reilly Media, 2022. 10. TensorFlow Developers, TensorFlow: An End-to-End Open Source Machine Learning Platform. Available: https://www.tensorflow.org
Publication Date
8/4/2026
Peer Review Type
Double-Blind Peer Review
Review Rounds: 2
Documents
Article Views
0
Total views since publication
PDF Downloads
10
Total downloads since publication
Geographic Reach
5
Digital Preservation: This article is permanently archived on Zenodo and complies with all data preservation standards across major indexing platforms.
Plagiarism Screening
This manuscript has been checked for plagiarism using TurnitinPlagiarismChecker software.
Conflict of Interest
The authors declare no conflict of interest related to this research work.
Funding Information
No funding information available for this research.
Related research in the same subject area
Loading similar articles...