AI-Driven Gesture Recognition and Multilingual Translation

 AI-Driven Gesture Recognition and Multilingual Translation

1Prof. Ravindra Patil
2Nikhil Suresh Yallaraddi, Prasannakumar Pattar,

Prateek Hadli, Pritam A Shetty
Assistant Professor Department of CSE,
Final Year Students Department of CSE
KLS Vishwanathrao Deshpande Institute of Technology

Haliyal
Uttara-Kannada District, Karnataka, India
rtp@klsvdit.edu.in

2vd21cs031@klsvdit.edu.in , 2vd21cs034@klsvdit.edu.in

2vd21cs035@klsvdit.edu.in , 2vd21cs036@klsvdit.edu.in

Abstract – This paper presents a real-time system designed to improve communication effectively for everyone with speech and hearing impairments through gesture-based language translation. This approach uses machine learning algorithms to interpret American Sign Language (ASL) hand gestures which is a universal sign language and convert them into both text and speech outputs. By integrating Mediapipe for landmark detection with a Convolutional Neural Network (CNN) for gesture classification, the system effectively identifies static hand signs and ensures robustness in diverse surrounding environment with proper lighting conditions. The recognized gestures are further translated into various languages using natural language processing techniques, followed by speech synthesis through text-to-speech tools. The implementation includes a user-friendly graphical interface, enabling live gesture capture and multilingual feedback, making the system both accessible and adaptable for real-world scenarios.