Signwave – Sign Language Recognition

Signwave - Sign Language Recognition

Prof. R.J.Dhanall*1, Aditya Kamble*2, Varad Kulkarni*3, Pranav Mane*4, Nayan Tambave*5, Siddhi Randive*6

Department of Computer Engineering, D Y Patil College Kasba Bawda Kolhapur, Maharashtra, India.

ABSTRACT

Manual interpretation of sign language presents significant challenges due to the complexity of gestures and variations in individual signing styles, creating communication barriers for the deaf and hard-of-hearing community. Traditional methods relying on human interpreters are not scalable for real-time applications. This research proposes an automated Sign Language Recognition (SLR) system using deep learning and action recognition to enable accurate, real-time translation of gestures into text or speech.

The system integrates Long Short-Term Memory (LSTM) networks with MediaPipe Holistic for spatiotemporal feature extraction. A diverse dataset covering 25 gestures was curated and augmented with variations in lighting, background, and signing speed.

The model architecture combines CNN-based spatial feature extraction with stacked LSTMs to capture temporal dependencies, achieving 87% accuracy on test data. The system was deployed as a Flask-backed web application with a React.js frontend, delivering real-time translation.

Comparative analysis shows superior performance over baseline models (ResNet-50: 84.3%, EfficientNet: 85.6%). The system demonstrates practical utility in educational and healthcare settings, with 89% user satisfaction in pilot testing. Future work will expand the gesture vocabulary, integrate Transformer architectures for continuous signing, and optimize for mobile deployment using TensorFlow Lite.

Keywords:LSTM, Action Recognition, MediaPipe, Sign Language Translation, Deep Learning, Real-Time Systems.