Enhancing Communication between Speech and Hearing Impaired People

Enhancing Communication between Speech and Hearing Impaired People

Dr Brindha S1,Ms  Sudha K2, Mr Amudhiniyan S3 , Mr Darshan S 4, Mr Jeganathkumar S5,              Mr Karthi S6,       Mr Jyothis James 7

1Head of the Department, Computer Networking, PSG Polytechnic College, Coimbatore

2Lecturer, Computer Networking, PSG Polytechnic College, Coimbatore

3,4,5,6,7 Students, Computer Networking, PSG Polytechnic College, Coimbatore

---------------------------------------------------------------------***---------------------------------------------------------------------

Abstract -Mute Mate is a novel video conferencing system that uses Artificial Intelligence and Real-Time communication technologies to bridge the communication gap between sign language users and verbal communicators. The system uses YOLOv11 for sign language detection, OpenAI's Whisper model for speech-to-text translation, and WebRTC for real-time lag-free video communication. It ensures seamless communication between users of different modes of communication. Large-scale testing demonstrates the system's remarkable accuracy, low latency, and effectiveness, demonstrating its potential to revolutionize accessibility and inclusivity in digital communication. This study highlights the growing prominence of artificial intelligence in assistive communication technologies and the potential of combining deep learning with real-time processing for a more inclusive future.

Key Words: Sign Language Recognition, Speech-to-Text Conversion, Real-Time Communication, YOLOv11, OpenAI Whisper, WebRTC