Sign Language Recognition System

Sign Language Recognition System

Trupti Deshmukh1, Arya Hariharan Nair2, Eshwariya Sapkal3  ,Mehnaz Uluwar4 ,Shubhangi Chavan5

1-5Department of Computer Engineering & Pillai College of Engineering(Autonomous), New Panvel, India

---------------------------------------------------------------------***---------------------------------------------------------------------

Abstract - Sign language is a visual language that conveys meaning through a combination of hand gestures, body language and facial expressions. Sign languages are used by people who are deaf or mute or are hearing impaired, to communicate with each other, as well as with hearing people who understand the sign language. To facilitate and aid the communication between a wide range of hearing, hard of hearing, or deaf individuals and those who don’t understand sign language, we propose a ML based sign language recognition system that uses feature extraction to identify patterns and make predictions about the user's hand characteristics captured through the webcam. We have proposed a system that utilises the convolutional neural network to recognize Fingerspelling gestures of the American Sign Language. We use Tensorflow and Keras to build and train the CNN model for gesture recognition and classification. The recognised alphabets can then be combined in order to form a coherent word. NLP techniques are applied to suggest words from the detected alphabets. The words can further be appended together to form a sentence.

Key Words: Deaf and Hard of Hearing Community(DHH), Convolutional Neural Network (CNN) , Sign Language