A Multilingual Adaptive Digital Speech Rehabilitation System with Real-Time Feedback

Aphasia is a neurogenic communication disorder that disrupts an individual’s ability to speak, understand, read, or write, commonly occurring after a stroke or traumatic brain injury. Traditional speech therapy, although effective, is often time consuming, expensive, and limited by linguistic and geographical accessibility. This paper presents a scalable and multilingual AI-driven speech therapy system integrating automatic speech recognition (ASR), text-to-speech (TTS), adaptive feedback, and an avatar-based phoneme–viseme synchronization module for English, Hindi, and Kannada. The system utilizes multilingual Wav2Vec2 embeddings, phoneme-level speech analysis, and ani mated lip-sync feedback generated using Rhubarb Lip Sync and Media Pipe to deliver multimodal and inclusive rehabilitation. Experimental results demonstrate ASR accuracy between 89% and 93% across the supported languages, along with high user engagement reflected by an average patient satisfaction rating of 4.6 out of 5 and positive therapist feedback. The modular architecture enables deployment in resource-constrained environments while maintaining clinical relevance through integration with standardized assessment metrics such as the Western Aphasia Battery-Aphasia Quotient. Overall, the proposed framework provides an accessible, cost-effective, and technologically enhanced approach to speech therapy for individuals with aphasia.