Intelligent Hand Sign Detection for Deaf and Mute People: A Multimodal Approach to Enhancing Communication through AI-Driven Gesture Recognition

Intelligent Hand Sign Detection for Deaf and Mute People:A Multimodal Approach to Enhancing Communicationthrough AI-Driven Gesture Recognition

Madhav D. Ingle1, Mahesh S. Suryawanshi2, Prajakta A. Shinde3, Sonali T. Waghmare4

1Department of Computer Engineering & JSPM’s JSCOE Pune, India

2Department of Computer Engineering & JSPM’s JSCOE Pune, India

3Department of Computer Engineering & JSPM’s JSCOE Pune, India

4Department of Computer Engineering & JSPM’s JSCOE Pune, India

---------------------------------------------------------------------***---------------------------------------------------------------------

Abstract - Intelligent hand sign detection systems offer a transformative solution for enhancing communication between deaf and mute individuals and the broader community. This paper presents a multimodal approach for hand sign detection that integrates advanced AI-driven gesture recognition techniques. By leveraging a combination of computer vision, deep learning, and sensor technologies, the proposed system is capable of accurately recognizing and interpreting a wide range of hand signs. The approach utilizes Convolutional Neural Networks (CNN) for image-based gesture recognition, while also incorporating real-time feedback through audio and visual cues for a more interactive experience. The system is designed to bridge communication barriers, allowing deaf and mute individuals to communicate seamlessly in various real-life scenarios. The integration of multimodal input, including motion sensors and voice synthesis, enhances the robustness of the system, making it adaptable to diverse environments. The results demonstrate the effectiveness of the proposed solution in improving communication accessibility, fostering inclusivity, and empowering individuals with speech and hearing impairments.

Keywords: Hand sign detection, Deaf and mute communication, Gesture recognition, Deep learning, Inclusive communication