Loading article...
Loading article...
Stream/Area:
-
Subject:
-
Paper ID:
IJSREM61852
Volume:
10 |
4 |
April 2026
Pages:
1-6
Publication Frequency:
Monthly (12 issues per year)
License:International Journal of Scientific Research in Engineering and Management follows the Creative Commons Attribution 4.0 (CC BY 4.0) License, allowing free use and distribution with proper attribution.
Dept. of Electronics and Communication Engineering
B.S. Abdur Rahman Crescent Institute of Science and Technology Chennai, Tamil Nadu, India
Dept. of Electronics and Communication Engineering
B.S. Abdur Rahman Crescent Institute of Science and Technology Chennai, Tamil Nadu, India
Dept. of Electronics and Communication Engineering
B.S. Abdur Rahman Crescent Institute of Science and Technology Chennai, Tamil Nadu, India
Dept. of Electronics and Communication Engineering
B.S. Abdur Rahman Crescent Institute of Science and Technology Chennai, Tamil Nadu, India
Communication remains a critical challenge for individuals with hearing and speech impairments in India, where Indian Sign Language (ISL) is the primary mode of interaction yet lacks widespread adoption. This paper presents a smart glove-based ISL translation system that integrates multimodal sensing with deep sequential learning for real-time gesture classification.The proposed system employs resistive flex sensors to capture finger joint articulation and an MPU-6050 IMU to acquire accelerometer and gyroscopic data for modeling hand orientation and motion dynamics. Sensor data is acquired via an ESP32 microcontroller, where preprocessing steps such as noise filtering, normalization, and temporal segmentation are performed prior to transmission.Temporal dependencies in gesture sequences are modeled using a Long Short-Term Memory (LSTM) network within Deep Learning, enabling effective learning of sequential patterns and motion trajectories. The model is trained on a custom ISL dataset and achieves a classification accuracy of 92.4%, outperforming Random Forest (88.7%), Support Vector Machine (79.3%), and K-Nearest Neighbors (74.5%).The system achieves an end-to-end inference latency of <150 ms, satisfying real-time processing requirements. Recognized gestures are mapped to textual outputs and synthesized speech via a React-based interface, with lightweight communication facilitated through the MQTT protocol.The proposed architecture demonstrates a robust, low-latency, and scalable assistive framework for ISL interpretation, with potential deployment in real-world assistive communication systems.
KEYWORDS
This paper presented a Smart Glove-Based Indian Sign Language Translator integrating ESP32-based wearable sensing, LSTM deep learning, IoT communication via MQTT, and a React-based web interface. The system achieves 92.4% gesture recognition accuracy with sub-150ms inference latency, enabling practical real-time ISL translation with both text and speech outputs. The dual-pathway architecture combining hardware glove inference via MQTT with camera-based MediaPipe recognition enhances versatility. The system demonstrated stable real-time performance with gesture confirmation latency under 0.5 seconds. Future work will focus on expanding the ISL vocabulary, implementing transformer-based sequence models, bilateral two-glove support, NLP-based sentence construction from gesture sequences, and edge deployment of the inference model directly on the ESP32 microcontroller.
[1] Hrithik T. H., Rhethika S., Akhil K. H., and Deepa K., "ML Assisted Sign Language to Speech Conversion Gloves for the Differently Abled," IEEE IATMSI, 2024. [2] X. Ji, Y. Zhao, and X. Yu, "Dataglove for Sign Language Recognition of People with Hearing and Speech Impairment via Wearable Inertial Sensors," IEEE Sensors Journal, 2023. [3] S. K. Patil and R. R. Deshpande, "Smart Glove Based Sign Language Recognition System Using Machine Learning," IEEE ICSSIT, 2022. [4] M. R. Islam and M. A. Hossain, "Wearable Sensor Based Sign Language Recognition Using Machine Learning," IEEE ICBECA, 2021. [5] P. K. Sharma and A. Verma, "Real-Time Indian Sign Language Recognition System Using Machine Learning Techniques," IEEE ICCCA, 2020. [6] A. Kumar, S. Mehta, and R. Jain, "Smart Wearable System for Sign Language Interpretation Using Machine Learning," IEEE ICAIS, 2023. [7] N. S. Rao and P. S. Reddy, "Hand Gesture Recognition Using Sensor Based Smart Glove and Machine Learning," IEEE ICECA, 2022. [8] S. Banerjee and A. Mukherjee, "Indian Sign Language Recognition Using Wearable Sensors and Machine Learning Algorithms," IEEE ICSPC, 2021. [9] S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997. [10] F. Zhang et al., "MediaPipe Hands: On-device Real-time Hand Tracking," arXiv:2006.10214, 2020. [11] M. Abadi et al., "TensorFlow: A System for Large-Scale Machine Learning," Proc. OSDI, 2016. [12] A. Banks and R. Gupta, "MQTT Version 3.1.1," OASIS Standard, 2014.
Publication Date
5/1/2026
Peer Review Type
Double-Blind Peer Review
Review Rounds: 2
Documents
Article Views
0
Total views since publication
PDF Downloads
3
Total downloads since publication
Geographic Reach
5
Digital Preservation: This article is permanently archived on Zenodo and complies with all data preservation standards across major indexing platforms.
Plagiarism Screening
This manuscript has been checked for plagiarism using TurnitinPlagiarismChecker software.
Conflict of Interest
The authors declare no conflict of interest related to this research work.
Funding Information
No funding information available for this research.
Related research in the same subject area
Loading similar articles...