A USER-CENTRIC SPEECH TO SIGN LANGUAGE ASSISTIVE SYSTEM FOR HEARING-IMPAIRED USERS

We built A2SL to fill a gap that existing assistive tools leave open: a practical, hardware-free way for hearing-impaired users to access spoken content in ISL form. The system handles three input types, achieves 89% direct sign matching, and preserves 100% of information through its fallback chain. Tamil and Hindi support extends its reach to a wider slice of India’s deaf community. The lightweight Django architecture means it can realistically be deployed at schools, clinics, or community centres without specialized infrastructure. There is plenty of room to grow from here — adding more signs through community sourcing, integrating transformer-based NLP for better accuracy, building bidirectional sign-to-speech support, creating a mobile version for low-bandwidth environments, and eventually allowing user profiles to adjust sign speed and presentation style to individual needs.

[1] M. Kumar, A. Natarajan, S. S. Visagan, T. Mahajan and P. S. Sreeja, “Enhanced Sign Language Translation Between American Sign Language and Indian Sign Language Using LLMs,” IEEE Access, 2025. [2] I. Marin, V. Babalian and S. Slutu, “Real-Time Sign Language-to-Speech Translation System with AI-Powered Wearable Technology,” IEEE, 2025. [3] C. Chandini and A. S. P. Katuru, “Audio-to-Indian Sign Language Translation System Using NLP and Deep Learning,” International Journal/Conference, 2025. [4] P. Antonowicz, D. Kasperek and M. Podpora, “Sign Language Recognition — Dataset Cleaning for Robust Word Classification in a Landmark-Based Approach,” IEEE Access, 2025. [5] R. Karthik, S. Vignesh and M. Harini, “Beyond Words: Speech to Sign Language Interpreter,” International Conference / IEEE, 2024. [6] A. A. Al-Shamayleh, R. Ahmad and M. Al-Khawaldeh, “A Real-Time Intelligent System Based on Machine-Learning Methods for Improving Communication in Sign Language,” IEEE / International Journal, 2024. [7] Y. Zhang, C. Chen and H. Liu, “Word-Level Sign Language Recognition with Multi-Stream Neural Networks Focusing on Local Regions and Skeletal Information,” IEEE, 2024. [8] R. Nivetha, S. Priyadharshini and K. Saravanan, “Speech to Indian Sign Language Translator,” International Conference / IEEE, 2024. [9] A. S. Raghavendra, P. K. Suresh and M. Anitha, “Speech to Sign Language Translation for Indian Languages,” IEEE Conference, 2023. [10] Z. Zhou, W. Li and Y. Xu, “Multi-Semantic Discriminative Feature Learning for Sign Gesture Recognition Using Hybrid Deep Neural Architecture,” IEEE Access, 2023. [11] W. Stokoe, D. Casterline, and C. Croneberg, A Dictionary of American Sign Language on Linguistic Principles. Gallaudet University Press, 1965. [12] R. Bowden et al., "Software Glove for Sign Language Interpretation," IEEE Trans. Syst., Man, Cybern., vol. 34, no. 3, pp. 1555–1567, 2004. [13] S. Koller, O. Zargaran, H. Ney, and R. Bowden, "Deep Sign: Robust Statistical Sign Language Recognition via Hybrid CNN-HMMs," in Proc. BMVC, 2016. [14] S. Stoll, N. Camgoz, S. Hadfield, and R. Bowden, "Sign Language Production Using Neural Machine Translation and GANs," in Proc. BMVC, 2018. [15] B. Bird and E. Klein, Natural Language Processing with Python. O'Reilly Media, 2009. [16] D. Jurafsky and J. H. Martin, Speech and Language Processing, 3rd ed. Pearson, 2022. [17] Google LLC, "Google Cloud Speech-to-Text Documentation," [Online]. Available: https://cloud.google.com/speech-to-text. [Accessed: Mar. 2025]. [18] Django Software Foundation, "Django Web Framework Documentation," [Online]. Available: https://docs.djangoproject.com. [Accessed: Mar. 2025]. [19] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in Proc. NAACL-HLT, pp. 4171–4186, 2019.  [20] N. C. Camgoz, S. Hadfield, O. Koller, H. Ney, and R. Bowden, "Neural Sign Language Translation," in Proc. IEEE CVPR, 2018.