Inclusive Communication: Leveraging AI for Sign Language Translation and Real-Time Audio Transcription

Inclusive Communication: Leveraging AI for Sign Language Translation and Real-Time Audio Transcription

Shwethashree G C1, Aditya T2, Neha Dayanand3, Rishin S4, Adi Sharan A5

1Shwethashree G C, Dept. of Computer Science and Engineering, JSS Science and Technology University,

2Aditya T, Dept. of Computer Science and Engineering, JSS Science and Technology University,

3Neha Dayanand, Dept. of Computer Science and Engineering, JSS Science and Technology University,

4Rishin S, Dept. of Computer Science and Engineering, JSS Science and Technology University,

5Adi Sharan A, Dept. of Computer Science and Engineering, JSS Science and Technology University

Abstract - Humans communicate through both natural language and body language, including gestures, facial expressions, and lip movements. While understanding spoken language is essential, recognizing sign language is equally important, especially for individuals with hearing impairments. Deaf individuals often struggle to communicate with those unfamiliar with sign language, making real-time translation systems invaluable. This paper proposes a real-time meeting platform that recognizes Indian Sign Language (ISL) gestures and converts them into text and speech, enabling smooth interaction between deaf and hearing individuals. The system uses image processing, computer vision, and deep learning—specifically Long Short-Term Memory (LSTM) networks—to analyze hand gestures from a live video stream. LSTM models effectively capture temporal patterns in gesture sequences, enhancing recognition accuracy. The identified gestures are then translated into text and synthesized into speech. This system aims to bridge communication gaps and improve accessibility for the hearing impaired.

Key words: Indian Sign Language (ISL), Gesture Recognition, LSTM, Real-Time Translation, Accessibility, Communication, Deep Learning, Speech Synthesis, Computer Vision, Virtual Meetings.