Student Attention Monitoring Using Uncertainty-Calibrated Quality-Adaptive Facial and Pose Fusion: An Empirical Feasibility Study

This paper presented ST-MAN, an implemented student-attention monitoring system whose main methodological contribution is UCQAF, a trainable fusion mechanism that conditions facial and pose weights on quality, uncertainty, temporal stability, disagreement, and modality availability. The system achieved 97.0% accuracy, 96.0% balanced accuracy, 96.0% macro-F1, and 98.0% weighted-F1. Cohen’s κ and MCC were 0.94 and 0.95, while ECE and Brier score were 0.03 and 0.04. On the public classroom benchmark, the proposed method exceeded the published 84.04% pose-CNN result. A representative participant-disjoint test run correctly classified 387 of 400 clips. Ablation and robustness analyses showed that facial and postural evidence were complementary, temporal context was particularly useful for drowsiness, and quality-adaptive fusion reduced degradation under blur, occlusion, and landmark loss. CPU-only execution reached 4.7 FPS, demonstrating feasibility for sampled-frame monitoring while identifying a clear need for deployment optimisation.

[1]     Y. Kartynnik, A. Ablavatski, I. Grishchenko, and M. Grundmann, “Real-time facial surface geometry from monocular video on mobile GPUs,” arXiv:1907.06724, 2019. [2]    S. Yan, Y. Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” in Proc. AAAI Conf. Artif. Intell., 2018. [3]     A. Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017. [4]     S. Vats, A. K. Gupta, I. Goel, and S. Chaurasia, “AI-driven student atten-tion monitoring using pose estimation and convolutional neural networks,” in Proc. IEEE World Conf. Applied Intelligence and Computing, 2025, pp. 140–145. [5]     D. X. H. Chew and T. H. Teo, “Real-time online learning attention tracker,” in Proc. IEEE Int. Conf. Teaching, Assessment and Learning for Engineering, 2022, pp. 410–414. [6]     M. A. Moeid and M. M. Haque, “Real-time student attention monitoring in classrooms using computer vision,” in Proc. Int. Conf. Computer and Information Technology, 2025, pp. 263–268. [7]     L. Han, X. Ma, M. Dai, and L. Bai, “A WAD-YOLOv8-based method for classroom student behavior detection,” Scientific Reports, vol. 15, Art. no. 9655, 2025, doi: 10.1038/s41598-025-87661-w. [8]     X. Sheng, S. Li, and S. Chan, “Real-time classroom student behavior detection based on improved YOLOv8s,” Scientific Reports, vol. 15, Art. no. 14470, 2025, doi: 10.1038/s41598-025-99243-x. [9]     L. Marquez-Carpintero et al., “DIPSER: A dataset for in-person student engagement recognition in the wild,” arXiv:2502.20209, 2025. [10]    S. Gupta, P. Kumar, and R. Tekchandani, “A multimodal facial cues based engagement detection system in e-learning context using deep learning approach,” Multimedia Tools and Applications, vol. 82, pp. 28589–28615, 2023.