Audio-to-Sign Language Translator

This paper presents the design and implementation of a real-time system for bridging the communication gap between the hearing and the deaf or hard-of-hearing. The proposed system, developed as a Python-based desktop application, leverages a comprehensive suite of open-source technologies to ensure economic feasibility and accessibility. It utilizes the OpenCV library for live video capture, hand gesture preprocessing, and feature extraction. For static sign classification, a K-Nearest Neighbors (KNN) model is employed, chosen for its efficiency and robust performance on user-generated datasets. The system integrates SpeechRecognition for audio-to-text conversion and pyttsx3 for synthesized speech output, enabling bidirectional communication. A distinguishing feature is the user-trainable module, which empowers individuals to dynamically expand the system’s vocabulary, thereby addressing a fundamental challenge of sign language diversity and data scarcity. The report details the system’s modular architecture and its operational methodology, including data capture, model training, and real-time recognition. We present the system’s performance metrics and discuss its a-priori design choices that favor usability and scalability. The findings demonstrate that a user-centric, economically viable system can provide an effective and inclusive communication tool, establishing a foundation for advanced future enhancements.