Chayan AI: Real-Time Multimodal Interview Analysis Using NLP, Computer Vision, and Deep Learning for Candidate Evaluation

This work presented  Chayan AI, an intelligent interview and candidate evaluation platform that integrates advanced artificial intelligence techniques to modernize and streamline the recruitment process. By combining Natural Language Processing, Machine Learning, Computer Vision, and Generative AI within a unified framework, the proposed system effectively bridges the gap between traditional human-driven interviews and fully automated, data- driven hiring systems. The developed platform enables end-to-end interview automation, from adaptive question generation to multimodal candidate analysis and final evaluation. By analyzing both verbal and non-verbal cues, the system provides a more comprehensive understanding of candidate performance compared to conventional methods. Experimental observations indicate improved consistency in evaluation, reduced bias, faster processing time, and enhanced user experience for both recruiters and candidates. The integration of generative AI allows dynamic and context-aware interviews, while machine learning models ensure objective and reliable scoring. The automated evaluation and reporting pipeline eliminates the need for manual assessment, significantly reducing recruiter workload and improving decision-making efficiency. Leveraging modern technologies such as React-based interfaces, Python backend frameworks, and scalable cloud infrastructure, the system is modular, extensible, and adaptable to various recruitment scenarios across industries. Despite its advantages, certain limitations remain. The accuracy of evaluation depends on the quality and diversity of training data, and real-time video analysis may vary across different hardware environments. Additionally, ethical considerations such as data privacy, fairness, and transparency must be carefully addressed through secure data handling, bias mitigation techniques, and explainable AI models. Looking ahead, future enhancements may include integration with job portals and professional networks, advanced emotion and personality analysis, multilingual interview support, and real- time cheating detection mechanisms. Incorporating explainable AI and continuous learning models can further improve trust and adaptability in the system. In summary, Chayan AI successfully demonstrates how a unified, multimodal AI framework can transform traditional hiring practices into a more scientific,  unbiased,  and  efficient  process.  The proposed system lays a strong foundation for next- generation intelligent recruitment platforms, enabling smarter decision-making and more equitable opportunities in the evolving digital workforce.

[1]     Sharma, P., & Gupta, N., 2025, AI-Based Automated Interview Systems for Smart Recruitment, International Journal of Engineering Research & Technology (IJERT), Volume 14, Issue 07. [2]   Wang, X., Li, Y., & Chen, Z., 2025, Intelligent Candidate Evaluation Using Multimodal Machine Learning, arXiv preprint arXiv:2503.11234. [3]   Kumar, A., Singh, R., & Verma, S., 2025, AI- Driven Hiring Platforms with Bias Reduction Techniques, IEEE International Conference on Smart Computing Systems, pp. 210-218. [4]    Dutta, S., et al., 2025, Automated Interview Analysis Using NLP and Computer Vision, IEEE Conference on Artificial Intelligence Applications (WACV), 2025. [5]  Liu, Y., Fang, Y., & Lin, Z., 2024, Multimodal AI Systems for Behavioral Analysis in Recruitment, arXiv preprint arXiv:2409.05221. [6]   Jang, Y., et al., 2024, Joint Speech and Visual Analysis for Intelligent Human Interaction Systems, Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. [7]  Sun, Y., et al., 2024, AI-Based Human Behaviour Understanding Using Multimodal Data, IEEE Access, Vol. 12, pp. 57288–57301. [8]     Chen, S., et al., 2024, Speech and Facial Expression Analysis for Automated Interviews, arXiv preprint arXiv:2401.09951. [9]  Yaman, D., et al., 2024, Video-Based Candidate Assessment with Deep Learning Models, European Conference on Computer Vision (ECCV), Springer, 2024. [10]   Biadsy, F., et al., 2024, Cross-Lingual Speech Understanding for AI Systems, arXiv preprint arXiv:2409.13910.