AI-Based Dysarthric Speech Recognition and Voice Assistance System

Dysarthria is a motor speech disorder caused by neurological impairments affecting the muscles responsible for speech production. Individuals with dysarthria typically experience slurred articulation, irregular speech rhythm, reduced vocal intensity, and inconsistent pronunciation patterns, making their speech difficult for both humans and machines to understand. Automatic Speech Recognition (ASR) systems have achieved high accuracy for typical speech; however, performance degrades significantly when applied to dysarthric speech. This paper presents DysVoice, an AI-based dysarthric speech assistance system that fine-tunes OpenAI’s Whisper Small model on the TORGO dysarthric speech dataset to produce accurate transcriptions of dysarthric speech, which are then converted to spoken output through a text-to-speech engine. The system is deployed on a Raspberry Pi 4 with a collar microphone for input, a physical press button for recording control, and a Bluetooth speaker for audio output, forming a fully portable and wearable assistive device. The proposed system was trained on 2,917 audio-transcript pairs from 6 dysarthric speakers and evaluated on 2 held-out speakers not seen during training. Experimental results demonstrate that the fine-tuned Whisper model achieves a Word Recognition Accuracy of 95.54% on speaker M04 (mild/moderate dysarthria) and 97.20% on speaker F03 (moderate dysarthria), significantly exceeding the target benchmark of 85% and outperforming conventional speech recognition approaches on dysarthric speech.