Cognitive AI-Based Emotional State Monitoring for Counselling Sessions

This paper presents a Cognitive AI-Based Emotional State Monitoring framework for counselling sessions that combines facial expression analysis, speech emotion recognition, and text-based emotion analysis. The proposed workflow integrates modality-specific predictions through late fusion and applies temporal analysis to monitor changes over the course of a session. A counsellor dashboard is used to organise the resulting indicators for professional observation. The framework is intended as an AI-based decision-support mechanism rather than an autonomous diagnostic system. The current source document defines the architecture, algorithm, evaluation measures, and an illustrative comparative table, but it does not provide sufficient experimental evidence to validate the reported percentage values. Therefore, actual dataset-based training, testing, ablation analysis, confusion matrices, and real-time validation are required before the performance claims can be treated as experimental findings. Future work should focus on validated multimodal datasets, temporal evaluation, explainability, privacy-preserving implementation, and evaluation with qualified counselling professionals. These steps can establish the practical usefulness and limitations of the proposed framework while preserving professional oversight.

[1] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016. [2] P. Ekman, “An argument for basic emotions,” Cognition and Emotion, 1992. [3] C. Busso et al., “IEMOCAP: Interactive emotional dyadic motion capture database,” Language Resources and Evaluation, 2008. [4] S. R. Livingstone and F. A. Russo, “The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS),” PLOS ONE, 2018. [5] J. Devlin et al., “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019. [6] T. Baltrušaitis, C. Ahuja, and L.-P. Morency, “Multimodal machine learning: A survey and taxonomy,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.