Sign language translator using AI

The proposed AI-Based Real-Time Sign Language Translator demonstrates the effective integration of artificial intelligence, computer vision, machine learning, and speech technology for accessible communication. The system captures hand gestures using a webcam and applies image preprocessing to improve the quality of the input. MediaPipe Hands is used to detect the hand and extract 21 important landmarks representing finger and joint positions. These landmark coordinates are converted into feature vectors and supplied to a trained machine-learning classifier such as Random Forest, SVM, or KNN for gesture prediction. The recognized sign is subsequently converted into text and optionally transformed into speech using a text-to-speech module. The evaluated system achieved 94% gesture recognition accuracy, with 94 correctly recognized gestures out of 100 test samples. Text conversion and speech conversion achieved 96% and 93% success rates, respectively, while the average processing time was approximately 0.18 seconds. These results demonstrate the feasibility of real-time gesture recognition using a lightweight landmark-based approach. The system can support communication in education, healthcare, workplaces, public services, and assistive applications. Future improvements can include dynamic gesture recognition, larger datasets, multilingual sign languages, improved environmental robustness, and mobile or edge-device deployment.

[1] S. T. Abd Al-Latief, S. Yussof, A. Ahmad, S. M. Khadim, and R. A. Abdulhasan, “Instant sign language recognition by WAR strategy algorithm based tuned machine learning,” International Journal of Networked and Distributed Computing, vol. 12, pp. 344–361, 2024, doi: 10.1007/s44227-024-00039-8. [2] F. M. Najib, “Sign language interpretation using machine learning and artificial intelligence,” Neural Computing and Applications, 2025. [3] A. Baihan, A. I. Alutaibi, M. Alshehri, and S. K. Sharma, “Sign language recognition using modified deep learning network and hybrid optimization: A hybrid optimizer (HO) based optimized CNN-SaLSTM approach,” Scientific Reports, vol. 14, Art. no. 26111, 2024, doi: 10.1038/s41598-024-76174-7. [4] J. Zhang, X. Bu, Y. Wang, H. Dong, Y. Zhang, and H. Wu, “Sign language recognition based on dual-path background erasure convolutional neural network,” Scientific Reports, vol. 14, Art. no. 11360, 2024, doi: 10.1038/s41598-024-62008-z. [5] “Recent advances on deep learning for sign language recognition,” Computer Modeling in Engineering & Sciences, 2024. [6] N. Shahin and L. Ismail, “From rule-based models to deep learning transformers architectures for natural language processing and sign language translation systems: Survey, taxonomy and performance evaluation,” Artificial Intelligence Review, 2024. [7] M. Islam, M. Aloraini, S. Aladhadh, et al., “Toward a vision-based intelligent system: A stacked encoded deep learning framework for sign language recognition,” Sensors, vol. 23, 2023. [8] N. F. Attia, M. T. F. S. Ahmed, and M. A. M. Alshewimy, “Efficient deep learning models based on tension techniques for sign language recognition,” Intelligent Systems with Applications, 2023. [9] B. Saini, D. Venkatesh, N. Chaudhari, T. Shelake, S. Gite, and B. Pradhan, “A comparative analysis of Indian sign language recognition using deep learning models,” Forum for Linguistic Studies, 2023. [10] K. Pandian, M. R. M. Azraai, M. K. Ismail, et al., “Sign language recognition using deep learning through LSTM and CNN,” Mekatronika, 2023.