Identifying the Emotion of a Paralyzed Person Using Deep Learning

This paper has presented a complete real-time patient emotion monitoring system designed to address the communication barrier experienced by paralyzed patients. The system integrates six purpose-built modules - image acquisition, preprocessing, emotion detection, monitoring and output, alert and notification, and database storage - into a cohesive pipeline that operates continuously and automatically with no manual intervention required. The CNN-based emotion classifier, trained on the FER-2013 dataset, achieves an overall accuracy of 64% across seven emotion classes, with substantially higher performance for the four clinically critical classes: happy at 90%, sad at 85%, angry at 88%, and neutral at 89%. These results, validated against a test set of over 28,000 samples, demonstrate that the system is reliably capable of distinguishing between positive and distress emotional states - which is precisely the discrimination that matters most in a patient monitoring context. The system's practical design choices - standard webcam hardware, SMS and email alerting through widely used APIs, interval-based notification throttling, and a persistent database layer - collectively ensure that the system is deployable in real clinical environments without requiring specialized infrastructure or technical expertise from caregivers. The non-invasive nature of facial expression-based monitoring is particularly important in the paralysis care context, where patient comfort and dignity are paramount. Future enhancements will focus on replacing the Haar Cascade face detector with a deep learning-based alternative such as MTCNN or RetinaFace to improve robustness under low-light and off-angle conditions. Transfer learning using pre-trained architectures such as ResNet-50 or VGGFace will be explored to improve classification accuracy, particularly for the underperforming disgust and fear classes. Multi-modal recognition incorporating physiological signals alongside facial expressions, and a dedicated mobile application for caregiver alert management, are also planned directions. Ultimately, this system represents a meaningful and deployable contribution toward using artificial intelligence to improve the quality of life for individuals with severe physical disabilities.

[1]   S. M. Quleema et al., "An Optimized CNN Architecture for Robust Facial Emotion Analysis," IEEE Access, 2025. [2]    J.-H. Kim et al., "Real-Time Emotion Classification and Prediction Using a Hybrid Facial Expression Recognition Model," IEEE Access, 2025. [3]      M. C. Gursesli et al., "Facial Emotion Recognition Through Custom Lightweight CNN Model," IEEE Access, 2025. [4]   D. Mocchahari et al., "To Detect and Analyze Continuous Emotion in Real-Time Online Classroom," IEEE Access, 2024. [5]   J. Yang et al., "Real-Time Facial Expression Recognition Based on Edge Computing," IEEE Access, 2024. [6]     J. Lan, X. Zhang, Y. Deng, and Y. Li, "Expression Recognition Based on Multi-Regional Coordinate Attention Residuals (MrCAR)," IEEE Access, 2023. [7]       A. Mollahosseini, B. Hasani, and M. H. Mahoor, "AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild," IEEE Transactions on Affective Computing, 2017. [8]   S. Li and W. Deng, "Deep Facial Expression Recognition: A Survey," IEEE Transactions on Affective Computing, 2020. [9]    B. Hasani and M. H. Mahoor, "Deep Facial Expression Recognition: A Survey," IEEE Transactions on Affective Computing, 2017. [10]    M. S. Hossain and G. Muhammad, "Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks," in Proc. CVPR Workshops, 2019.