Visual Speech Recognition Using Lips Movement

Visual Speech Recognition Using Lips Movement

Dr.  A. B. Gavali1, Pooja Shankar Ghugarkar2, Supriya Raghunath Khatake3,

Rajanandini Anil Kothawale4

1,2,3,4 S. B. Patil College of Engineering, Indapur, Computer Engineering

---------------------------------------------------------------------***---------------------------------------------------------------------

Abstract - The audio-visual speech recognition technology seeks to increase noise-robustness in mobile contexts by extracting lip movement from side-face pictures. Prior bimodal speech recognition algorithms depended on frontal face (lip) photos, but because users must be able to speak while holding a device in front of their face, these techniques are difficult for users to utilise. Our suggested method records lip movement using a tiny camera built into a smartphone, making it more practical, simple, and natural. Additionally, this technique effectively prevents a decline in the input voice's signal-to-noise ratio. (SNR). For CNN-based recognition, visual properties are extracted via optical-flow analysis and merged with audio data.

Key Words:  Convolutional Neural Network, Deep Learning, Image.