Lip Reading Using Deep Learning Techniques

Lip Reading Using Deep Learning Techniques

Lakshmisha S K1, Mohammed Dhayan Ahmed2, Srivatsa H3, Bavith Raj

1Professor, 2,3,4Student 1,2,3,4 Department of CSE

1,2,3,4Presidency University, Bengaluru, India

Abstract- Lip reading, or the task of interpreting speech by observing lip movements visually, has been a longstanding problem in computer vision and speech processing. Handcrafted feature-based traditional approaches tend to fail because they are prone to visual noise and depend on handcrafted features. This work introduces a deep learning-based method for real- time lip reading with Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks. The model is learned over a bespoke dataset gathered with webcam input and MediaPipe Face Mesh for precise mouth area extraction. The final system has the capability of real-time prediction of words spoken through video input, giving a solid ground for silent speech recognition system.

Keywords: Lip Reading, Deep Learning, CNN, LSTM, MediaPipe, Real-Time Speech Recognition, Computer Vision