IntervuAI: Cognitive Insight System for Skill and Behaviour Analysis

This paper presented IntervuAI, a comprehensive multimodal AI-powered interview simulation and assessment platform. By integrating resume intelligence through spaCy and BERT, personalized question generation through LangChain with Grok API and Ollama, real-time speech transcription through Whisper API, CNN-BiLSTM-based emotion detection, BERT-based sentiment and behavioral analysis, and a Weighted Performance Score Model, the system delivers holistic and data-driven interview performance assessment. Experimental evaluation demonstrates 87.3% emotion classification accuracy, 91.2% sentiment analysis accuracy, and a Pearson correlation of 0.81 between automated performance scores and expert human evaluator ratings. These results confirm the viability of the multimodal evaluation pipeline for automated interview assessment and validate the design decisions across all system components. Future work will extend IntervuAI with active facial expression analysis and eye-contact tracking through webcam integration, multi-language interview support, AI avatar interviewers for more immersive simulation, adaptive learning systems that dynamically adjust preparation plans based on longitudinal performance trajectories, and enterprise-level deployment infrastructure for organizational recruitment workflows. The successful integration of diverse AI subsystems into a cohesive, end-to-end platform demonstrates the practical viability of multimodal automated interview assessment and establishes a replicable architectural blueprint for future intelligent recruitment and training support systems.

[1] J. Geathers, Y. Hicke, C. Chan, N. Rajashekar, J. Sewell, S. Cornes, R. F. Kizilcec, and D. Shung. Benchmarking Generative AI for Scoring Medical Student Interviews in OSCEs. arXiv preprint arXiv:2501.13957, Jan. 2025. [Online]. [2] L. G´omez-Zaragoz´a, R. del Amor, M. J. Castro-Bleda, V. Naranjo, M. Alca˜niz Raya, and J. Mar´ın-Morales. EMOVOME: A Dataset for Emotion Recognition in Sponta neous Real-Life Speech. arXiv preprint arXiv:2403.02167, Mar. 2024. [Online]. [3] R. Pandey, K. Das, and M. Agarwal. Adaptive Question Generation with GPT-based LLMs for Skill Assessment. Procedia Computer Science, vol. 234, pp. 85–94, Jun. 2024. [Online]. [4] S. Cai, J. Zhang, K. Bao, C. Gao, Q. Wang, F. Feng, and X. He. Agentic Feedback Loop Modeling Improves Recommendation and User Simulation. Proc. 48th Int. ACM SIGIR Conf. Research and Development in Information Retrieval (SIGIR ’25), Padua, Italy, pp. 1–10, July 2025. [5] D. Li et al. EALD-MLLM: Emotion Analysis in Long-Sequential and De-Identity Videos with Multi-Modal Large Language Model. arXiv preprint arXiv:2405.00574, May 2024. [Online]. [6] Z. Wan, Z. Qiu, Y. Liu, and W.-Q. Zhang. Metadata-Enhanced Speech Emotion Recognition: Augmented Residual Integration and Co-Attention in Two-Stage Fine Tuning. arXiv preprint arXiv:2412.20707, Dec. 2024. Accepted for IEEE ICASSP 2025. [Online]. [7] Nofal, Abdullah; Ali, Hassan; Hadi, Muhammad; Ahmad, Aizaz; Qayyum, Adnan; Johri, Aditya; Al-Fuqaha, Ala; Qadir, Junaid. AI-Enhanced Interview Simulation in the Metaverse: Transforming Professional Skills Training through VR and Genera tive Conversational AI. 2025. [8] Roy, Pradeep; Chowdhary, Sarabjeet; Bhatia, Rocky. A Machine Learning Approach for Automation of Resume Recommendation System. 2025. [9] Wang, Z.; Patel, Ananya; Roberts, J.; Singh, Kavita. LLM-Based Automated Skill Assessment Using Speech and Behavioral Cues. 2025. [10] Lee, Minho; Gupta, R.; Hernandez, Sofia; Kim, Y. Multimodal Emotion Recognition for Real-Time Human–AI Interaction. 2025