SignSense AI: Transformer Based Indian Sign Language Recognition Using Key-Point Features

Deaf and hard-of-hearing communities rely on sign languages as their primary mode of communication, yet automated systems capable of interpreting these languages accurately and efficiently remain limited — particularly for Indian Sign Language. This work presents a gesture recognition architecture that departs from conventional image-based pipelines by operating entirely on structured body and hand landmark sequences extracted using MediaPipe. A Transformer encoder processes these sequences, exploiting parallel self-attention to capture the temporal dependencies that define meaningful signing gestures. The system is benchmarked on the INCLUDE dataset across two vocabulary scales: a 150-class configuration and the full 263-class collection. On the smaller configuration, the model attains 94% classification accuracy; on the full vocabulary, it achieves 90% — outperforming CNN and hybrid LSTM-GRU baselines in both settings. The compact landmark representation keeps inference computationally tractable, making the system a viable candidate for real-time assistive communication applications. These findings suggest that pairing geometric feature extraction with attention-based sequence modeling offers a scalable and practically deployable path toward ISL recognition.