Sign Language Detection Model Using Action Recognition LSTM and Deep Learning

Sign Language Detection Model Using Action Recognition LSTM and Deep Learning

Sourabh Pal1, Tauseef Alam2, Priyanshu Singh3, Raju4, Akash Chowdhury5

1Assistant Professor, Dept of CSE (Email id: iamtraj1@gmail.com)

2B. Tech. (CSE) Research Scholar, Department of CSE (Email id: mdtauseefalam123456@gmail.com)

IIMT College of Engineering, Greater Noida, UP, India

Abstract: Sign language serves as a vital communication medium for the Deaf and Hard of Hearing (DHH) community, yet action recognition by computational systems remains challenging. It represents an approach for sign language detection using action recognition principles through a Long Short-Term Memory (LSTM) and deep learning model. By leveraging the temporal dynamics and sequential nature of sign language gestures, the LSTM model is trained to identify and classify signs from video data accurately. The proposed system processes the video sequences to extract key frame features, which are input into the LSTM network, designed to capture the temporal dependencies and nuanced movements characteristic of sign language. A comprehensive dataset comprising diverse sign language gestures is utilized to train and evaluate the model. Experimental results demonstrate that the LSTM-based approach achieves high accuracy in sign language detection, outperforming traditional static frame-based methods. The system's performance is evaluated through various metrics, including precision, recall, and F1-score, showcasing its robustness in real-world scenarios. In this project, our research aims to create a cognitive system that is sensitive and reliable so that persons with hearing and speech impairments may utilize it in day-to-day applications.

Keywords: Image processing, sign motion, sensors.