Thermal-Aided Real Time Sign Interpretation: A Comprehensive Review

Thermal-Aided Real Time Sign Interpretation: A Comprehensive Review

Krishna Swaroop A1,Prajwal2, Pranith B H3, Prajwal H R4, Benaka Raj5

Department of Information Science Engineering, Malnad College of Engineering, Hassan,

Karnataka, India – 573201

Abstract - Sign language recognition (SLR) systems play a crucial role in making communication accessible for the Deaf and hard-of-hearing community. Unlike traditional RGB-based methods that often struggle with lighting issues and can be intrusive, thermal imaging provides a strong, contactless, and privacy friendly alternative. This survey dives into the latest advancements in thermal SLR, highlighting deep learning techniques like CNNs, RNNs, Transformers, and hybrid models. Thermal systems naturally tackle lighting challenges while ensuring user anonymity, with CNN-RNN architectures achieving over 95% accuracy on static signs and attention-based models facilitating continuous recognition. Real-time applications on embedded devices like the NVIDIA Jetson and Raspberry Pi show impressive low-latency performance. However, there are still hurdles to overcome, such as handling occlusions, limited datasets, and generalizing across different signers. Exciting trends are emerging, including edge-optimized SLR for wearables, multimodal fusion (combining thermal and EMG), and support for multiple languages, all while keeping fairness, interpretability (using tools like Grad-CAM/SHAP), and privacy in mind. Deep learning-enhanced thermal SLR shines in low-visibility situations, with YOLO and CNN-based models leading the charge in real-time applications. The combination of thermal imaging and AI offers a scalable and efficient route for accessible sign language translation, paving the way for future research in inclusive communication technologies.

Keywords: Thermal Imaging, Sign Recognition, Deep Learning, Convolutional Neural Networks, YOLO, Gesture Recognition.