Sign Language Detection Using Action Recognition LSTM Deep Learning Model

Sign Language Detection Using Action Recognition LSTM Deep Learning Model

Purushotam Kr. Singh1, Sachin Kumar2, Yuvraj Kar Pathak3

1Btech-CSE Student, IIMT College of Engg, Greater Noida, UP, India

Purushotamsingh501@gmail.com1

2 Btech-CSE Student, IIMT College of Engg, Greater Noida, UP, India

Kumarsachin6150@gmail.com 2

3 Btech-CSE Student, IIMT College of Engg, Greater Noida, UP, India

Karpathakyuvraj2003@gmail.com 3

Guide: Prof. Badal Bhushan

Assistant Professor, Department of CSE

Department Of Computer Science and Engineering

IIMT College Of Engineering, Greater Noida

ABSTRACT

Sign language serves as a vital communication medium for the Deaf and Hard of Hearing (DHH) community, yet its recognition by computational systems remains a complex challenge. This research paper presents a novel approach to sign language detection utilizing action recognition principles through a Long Short-Term Memory (LSTM) deep learning model. Leveraging the temporal dynamics and sequential nature of sign language gestures, the LSTM model is trained to accurately identify and classify signs from video data.

The proposed system processes video sequences to extract key frame features, which are then input into the LSTM network. The model's architecture is designed to capture the temporal dependencies and nuanced movements characteristic of sign language. We utilize a comprehensive dataset comprising diverse sign language gestures to train and evaluate the model.

Our experimental results demonstrate that the LSTM-based approach achieves high accuracy in sign language detection, outperforming traditional static frame-based methods. The system's performance is evaluated through various metrics, including precision, recall, and F1-score, showcasing its robustness in real-world scenarios.

Keywords: Gesture Recognition, Deep Learning(DL), Sign Language Recognition(SLR), TensorFlow, Matplotlib, Mediapipe, opencv-python, numpy.