A Survey on Sign Language Recognition with Video Chat

A Survey on Sign Language Recognition with Video Chat

           Mrs. Kavyashree J                                               Arshiya M                                                            Mirza Akeel Abbas Baig

CSE dept.                                                                    CSE dept.                                                                  CSE dept.

Rajiv Gandhi Institute of Technology                   Rajiv Gandhi Institute of Technology                Rajiv Gandhi Institute of Technology

Bangalore,India                                                        Bangalore,India                                                       Bangalore,India

kavya.jshankar@gmail.com                                   arshi9742@gmail.com                                            akeelabbas29@gmail.com

           Nida Haleema                                                      Noor Mohammed Masood

CSE dept.                                                                      CSE dept.

Rajiv Gandhi Institute of Technology                     Rajiv Gandhi Institute of Technology

Bangalore,India                                                           Bangalore,India

nidahaleema28@gmail.com                                    mhdnoor06@gmail.com

Abstract—

The use of sign language is a crucial tool for improving communication between hearing-impaired people and the general public. SLR (Sign Language Recognition) systems in the past have been sophisticated and challenging to train. However, in this research, we provide a novel method that makes use of SSD MobileNet V2 FPNLite 320x320 pre-trained models and object recognition based on TensorFlow's object detection. By enabling the identification and detection of a set of images, this method streamlines the training process. The suggested system will be trained and evaluated using 10 to 15 different American Sign Language symbols. A fundamental social skill used to exchange information is communication. It is frequently used to express oneself and to fulfill fundamental human needs including the desire for protection, safety, and connection. Several stages, diverse methods, and distinct consequences are used in this procedure. It typically refers to a two-way exchange of information in the local vicinity of touch. Information flows far more easily when people are speaking the same language than when they are speaking languages that are from distinct language families. In order to facilitate video chat communication between signers and non-signers, our proposed sign language recognition system is specifically created. Each peer can see and hear the other during a video conversation thanks to their individual cameras and microphones. Nevertheless, using our method, a specific peer can also view the indicators that the other peer on the other end of the video chat exhibits or copies. Our system employs object detection to recognize and track the signer's hand motions in real-time in order to accomplish this. The technology then overlays a graphic of the detected sign onto the non-video signer's feed. This overlay is positioned so that it does not obstruct the view of the signer by the non-signer and is not obtrusive. By using contemporary technology, our suggested solution is made to enable distant and two-way communication between signers and non-signers. With the help of our system, video chat communication is improved, allowing signers and non-signers to communicate and interact more effectively.

Index Terms— SLR(Sign Language Recognition), Video Chat, Object Detection, SSD MobileNet.