Real-Time Facial Emotion Detection System Using OpenCV and PyTorch for Intelligent Music Recommendation

This paper has presented the design, implementation, and comprehensive evaluation of a real-time facial emotion detection system integrated with an emotion-driven music recommendation engine. The work makes a concrete and reproducible contribution to the affective computing literature by demonstrating that accurate, real-time, privacy-preserving emotion recognition is achievable on standard consumer-grade CPU hardware — a practically significant result given the gap between the high computational requirements of recently reported benchmarks and the hardware accessible to most end users. The proposed CNN architecture was observed to achieve 68.4% accuracy on the FER-2013 PrivateTest benchmark with approximately 1.2 million parameters and 42 millisecond end-to-end pipeline latency, corresponding to 23.8 frames per second throughput on an Intel Core i5 CPU without GPU acceleration. These results place the proposed model at the most favorable position in the accuracy-efficiency Pareto frontier among CPU-deployable approaches, showing competitive performance relative to Mini-Xception by 2.4 percentage points while maintaining comparable real-time performance. The ablation study provides rigorous quantitative evidence that data augmentation (5.1% contribution), batch normalization (4.5%), class weighting (3.7%), dropout regularization (2.8%), global average pooling (2.1%), and four-block depth (2.6%) are each independently and meaningfully beneficial, guiding future practitioners in constructing efficient emotion recognition architectures. The mathematical formulations provided throughout the paper — for convolution, batch normalization, global average pooling, weighted cross-entropy loss, Adam optimization, softmax activation, and Russell circumplex distance — offer a complete and self-contained technical specification of the system that can be directly implemented and reproduced. The detailed confusion matrix analysis reveals the specific emotion pair confusions (Fear/Surprise, Sad/Neutral, Angry/Sad) that represent the primary remaining accuracy challenges, with direct implications for the music recommendation experience and for future model improvement strategies. The emotion-driven music recommendation module indicates that the Russell Circumplex Model of Affect provides a psychologically principled, computationally efficient, and interpretable basis for mapping discrete emotion categories to music mood regions in the valence-arousal space. The temporal smoothing mechanism ensures perceptually stable recommendation behavior without sacrificing responsiveness to genuine emotional transitions, and the fully local processing architecture supports user data privacy without requiring modification of the underlying computational approach. Looking ahead, the most impactful improvements to the system would be achieved through replacement of the Haar Cascade face detector with a lightweight neural detector such as BlazeFace, incorporation of temporal modeling to exploit expression dynamics, and introduction of reinforcement learning-based personalization for the music recommendation component. Demographic fairness evaluation and mitigation represent an ethical imperative for any future deployment beyond single-user personal entertainment contexts. In aggregate, this work indicates that the goals of emotional responsiveness, computational efficiency, and user privacy in interactive systems are not mutually exclusive. With careful architectural design and training methodology, real-time affective computing can be brought out of the GPU-equipped laboratory and into everyday devices accessible to all users.

[1] R. W. Picard, Affective Computing. Cambridge, MA, USA: MIT Press, 1997. [2] P. Ekman, "Facial expression and emotion," American Psychologist, vol. 48, no. 4, pp. 384–392, 1993. [3] P. Ekman and W. V. Friesen, "Constants across cultures in the face and emotion," J. Personality Social Psychol., vol. 17, no. 2, pp. 124–129, 1971. [4] Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436–444, 2015. [5] P. N. Juslin and J. A. Sloboda, Eds., Music and Emotion: Theory and Research. Oxford, UK: Oxford University Press, 2001. [6] Y.-H. Yang and H. H. Chen, "Prediction of the distribution of perceived music emotions using discrete samples," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 7, pp. 2184–2196, 2011. [7] Z. Zhang, "Facial expression analysis under partial occlusion: A survey," ACM Comput. Surv., vol. 54, no. 9, pp. 1–36, 2021. [8] F. Ricci, L. Rokach, and B. Shapira, Eds., Recommender Systems Handbook, 2nd ed. New York, NY, USA: Springer, 2015. [9] T. Ojala, M. Pietikainen, and T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971–987, 2002. [10] N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE CVPR, 2005, pp. 886–893.