SpeechVision : Multimodal Personal Feedback System Using Whisper AI and MediaPipe for Real-Time Speech and Visual Performance Evaluation

The proposed SpeechVision framework demonstrates the potential of integrating multimodal AI analysis with generative feedback systems for personalized communication coaching. By combining speech-to-text transcription (Whisper AI), facial emotion recognition (MediaPipe + OpenCV), and AI-driven argument evaluation (Gemini API), the system provides accurate, real-time, and actionable feedback on speech content, delivery, tone, and emotional expressiveness. This approach offers a scalable, cost-effective, and self-guided solution for improving public speaking and persuasive communication skills, reducing dependency on in-person mentors while maintaining high-quality coaching.

[1]   A. J. Moshayedi et al., "Smart Farming Solutions: A User-Friendly GUI for Maize Tassel Estimation Using YOLO With Dynamic and Fixed Labelling, Featuring Video Support," IEEE Access, vol. 13, pp. 57809–57833,                          2025,    doi: 10.1109/ACCESS.2025.3554984. [2]   T. S. Delwar, S. Mukhopadhyay, A. Kumar, M. Singh, Y.-w. Lee, J.-Y. Ryu, and A. S. M. S. Hosen, "Real-Time Farm Surveillance Using IoT and YOLOv8 for Animal Intrusion Detection," Future Internet, vol. 17, no. 2, p. 70, 2025, doi: 10.3390/fi17020070. [3]     K. V. Reddy et al., "Edge AI in Sustainable Farming: Deep Learning-Driven IoT Framework to Safeguard Crops From Wildlife Threats," IEEE Access, vol. 12, pp. 77707–77723, 2024, doi: 10.1109/ACCESS.2024.3406585. [4]      M. Ayaz, M. Ammad-Uddin, Z. Sharif, A. Mansour, and E.-H. M. Aggoune, "Internet-of-Things (IoT)-Based Smart Agriculture: Toward Making the Fields Talk," IEEE Access, vol. 7, pp. 129551–129583, 2019, doi: 10.1109/ACCESS.2019.2932609. [5]   D. Adami, M. O. Ojo, and S. Giordano, "Design, Development and Evaluation of an Intelligent Animal Repelling System for Crop Protection Based on Embedded Edge-AI," IEEE Access, vol. 9, pp. 132125–132139,                        2021,    doi: 10.1109/ACCESS.2021.3114503. [6]   J. Zhang, Y. Yuan, and Q. Wang, "Night Person Re-Identification and a Benchmark," IEEE Access, vol. 7, pp. 95496–95504, 2019, doi: 10.1109/ACCESS.2019.2929854. [7]   M. S. Farooq, S. Riaz, A. Abid, K. Abid, and M. A. Naeem, "A Survey on the Role of IoT in Agriculture for the Implementation of Smart Farming," IEEE Access, vol. 7, pp. 156237–156271, 2019, doi: 10.1109/ACCESS.2019.2949703. [8]  Z. J. Khow, Y.-F. Tan, H. A. Karim, and H. A. A. Rashid, "Improved YOLOv8 Model for a Comprehensive Approach to Object Detection and Distance Estimation," IEEE Access, vol. 12, pp. 63754–63767,      2024,      doi:10.1109/ACCESS.2024.3396224. [9]       A. A. AlZubi and K. Galyna, "Artificial Intelligence and Internet of Things for Sustainable Farming and Smart Agriculture," IEEE Access, vol. 11,   pp.   78686–78692,   2023,   doi:10.1109/ACCESS.2023.3298215. [10]   S. Lee, H. Ahn, J. Seo, Y. Chung, D. Park, and S. Pan, "Practical Monitoring of Undergrown Pigs for IoT-Based Large-Scale Smart Farm," IEEE Access, vol. 7, pp. 173796–173810, 2019, doi: 10.1109/ACCESS.2019.2934745. [11]    N. Fourati, A. Barkar, M. Dragée, L. Danthon-Lefebvre, and M. Chollet, "Probing Experts’ Perspectives on AI-Assisted Public Speaking Training," arXiv preprint arXiv:2507.07930, 2025. [12]     S. Padia et al., "Enhancing Public Speaking Skills Through AI-Powered Analysis and Feedback," Educational Administration: Theory and Practice, vol. 30, no. 5, pp. 15191–15199, 2024.