Interactive Communication System for Deaf and Mute Individuals Using Gesture and Speech Recognition

Interactive Communication System for Deaf and Mute Individuals Using Gesture and Speech Recognition

Shyamala S C, Athmeeya G M, Megha K E, Nayana H K, Sangeetha M S

Dept. of E&C Engineering

PES Institute Of Technology &    Management

Shimoga-577204

Abstract—Deaf and mute individuals often face challenges in communicating with others who are unfamiliar with sign language. This paper proposes a two-way communication system that bridges this gap by converting sign language gestures into text and speech and vice versa. The system uses a vision-based gesture recognition approach for translating hand movements into corresponding text and voice outputs. Additionally, spoken or typed text is converted into visual or haptic feedback to facilitate two-way interaction. The methodology employs advanced image processing algorithms, Support Vector Machines (SVM) for classification, and text-to-speech (TTS) and speech-to-text (STT) systems for seamless communication. Experimental results demonstrate a recognition accuracy of over 95% for common signs, making the system effective and reliable.

Keywords—Sign Language, Gesture Recognition, SVM, Text-to-Speech, Speech-to-Text, Two-Way Communication