Multilingual AI Translator with POS Analysis

This project introduced a multilingual AI-driven system for translation and linguistic analysis that can handle text and audio inputs in English, Hindi, and Kannada. The suggested system effectively combines speech recognition, multilingual translation, and Part-of-Speech (POS) tagging within a cohesive microservice-oriented framework. The platform delivers an effective and user-friendly setting for real-time language processing applications by integrating a React frontend, a Spring Boot backend, and a FastAPI AI service. The system employs advanced transformer-based and deep learning models to enhance the accuracy of speech transcription and multilingual translation. Audio recordings are transformed into text via the Whisper speech recognition model, whereas the NLLB transformer model manages multilingual translation. Linguistic analysis utilizes the Stanza NLP toolkit to identify grammatical categories in translated sentences. Experimental evaluation showed that the suggested framework functions efficiently for multilingual speech and text processing activities. The system effectively managed various types of user inputs while producing organized translations and POS tagging results with quick response times. The modular microservice architecture also enhances scalability, maintainability, and adaptability for future improvements.

[1]     A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust Speech Recognition via Large-Scale Weak Supervision,” Proceedings of the 40th International Conference on Machine Learning (ICML), vol. 202, pp. 28492–28518, 2023. [2]     N. Costa-jussà, M. Cross, P. Çıbıkdiken et al., “No Language Left Behind: Scaling Human-Centered Machine Translation,” Transactions of the Association for Computational Linguistics (TACL), vol. 10, pp. 1439–1457, 2022. [3]     Y. Liu, X. Yang, J. Zhang, Y. Xi, and D. Qu, “TAML-Adapter: Enhancing Adapter Tuning Through Task-Agnostic Meta-Learning for Low-Resource Automatic Speech Recognition,” IEEE Signal Processing Letters, vol. 32, pp. 636–640, 2025. [4]     S. Yusuyin, T. Ma, H. Huang, W. Zhao, and Z. Ou, “Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 33, pp. 1440–1453, 2025. [5]     A. M. Samin, M. H. Kobir, M. M. S. Rafee, M. F. Ahmed, M. Hasan, P. Ghosh, S. Kibria, and M. S. Rahman, “BanSpeech: A Multi-Domain Bangla Speech Recognition Benchmark Toward Robust Performance in Challenging Conditions,” IEEE Access, vol. 12, pp. 34527–34538, 2024. [6]     T. Hori, M. Kocour, A. Haider, E. McDermott, and X. Zhuang, “Delayed Fusion: Integrating Large Language Models into First-Pass Decoding in End-to-End Speech Recognition,” Proc. IEEE ICASSP, pp. 1–5, 2025. [7]     S. Ahmed, M. Singh, S. Bhattacharyya, and D. Coyle, “Decoding Neural Activity for Part-of-Speech Tagging,” Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 3079–3084, 2024. [8]     E. Dunbar, N. Hamilakis, and E. Dupoux, “Self-Supervised Language Learning from Raw Audio: Lessons from the Zero Resource Speech Challenge,” IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1211–1226, 2022. [9]     T. Dalai, T. K. Mishra, and P. K. Sa, “Part-of-Speech Tagging of Odia Language Using Statistical and Deep Learning-Based Approaches,” arXiv preprint arXiv:2207.03256, 2022. [10]   G. Wan, T. Mao, J. Zhang, H. Chen, J. Gao, and Z. Ye, “Grammar-Supervised End-to-End Speech Recognition with Part-of-Speech Tagging and Dependency Parsing,” Applied Sciences, vol. 13, no. 7, pp. 4243–4258, 2023.