Real-Time Bidirectional Translation System Between Text and Indian Sign Language Using Deep Learning and NLP Techniques

Real-Time Bidirectional Translation System Between Text and Indian Sign Language Using Deep Learning and NLP Techniques

1st Niyati V. Gaonkar

2nd Vishal R. Gori

3rd Anket G. Kadam

Dept. of Information Technology

V.E.S.I.T, Mumbai, India
niyatig26@gmail.com
vishalgori2208@gmail.com
anketkadam61@gmail.com

4th Soham H. Nimbalkar

 5th Mr. Manoj Sabnis

Dep. HOD, Dept. of Information Technology V.E.S.I.T, Mumbai, India

Abstract—In this paper, we present a real-time translation system that bridges the communication gap between the hearing and non-hearing communities. Our system converts English text to Indian Sign Language (ISL) and vice versa, using Natural Language Processing (NLP) techniques and deep learning-based gesture recognition. The system supports video-based gesture recognition for ISL and provides accurate text translations in real-time. This study addresses the technical challenges involved, including feature extraction from gestures and translating com- plex ISL sentences using neural networks like LSTM.

Keywords- Indian Sign Language (ISL), Sign Language Translation, Gesture Recognition, Deep Learning, LSTM Model, Mediapipe Holistic, Text-to-Sign Conversion, Dy- namic Gesture Segmentation, Fingerspelling, Natural Lan- guage Processing (NLP), Pose Estimation, Hand Landmark Tracking, Real-Time Sign Language Recognition, Data Aug- mentation, Accessibility Technology