Visual Speech Recognition Using Lips Movement

The audio-visual speech recognition technology seeks to increase noise-robustness in mobile contexts by extracting lip movement from side-face pictures. Prior bimodal speech recognition algorithms depended on frontal face (lip) photos, but because users must be able to speak while holding a device in front of their face, these techniques are difficult for users to utilise. Our suggested method records lip movement using a tiny camera built into a smartphone, making it more practical, simple, and natural. Additionally, this technique effectively prevents a decline in the input voice’s signal-to-noise ratio. (SNR). For CNN-based recognition, visual properties are extracted via optical-flow analysis and merged with audio data.