Lip Reading Using Machine Learning and Neural Networks

Lip Reading Using Machine Learning and Neural Networks

Mr. Shivaraj B G1, Akash2, Abhishek Ashokan Nambiar3, Harith Harish4, Shamir L C5

Student, Dept. of Computer Science & Engineering, Mangalore Institute of Technology & Engineering, Moodabidre, India2,3,4,5

Assistant Professor, Dept. of Computer Science & Engineering, Mangalore Institute of Technology & Engineering, Moodabidre, India1

Abstract: Lip reading, often overlooked in communication, involves visually interpreting lip movements to understand spoken words. This process entails recognizing lip positions and movements, organizing them into sound sequences, and decoding sentences. Machine learning can play a crucial role by training models on labeled datasets of lip movements and corresponding phonemes. These models can then extract features from new lip movements to classify spoken phonemes. Such technology can assist individuals with hearing impairments by improving speech recognition in noisy environments and aid security forces in situations lacking audio records. By integrating innovative technologies seamlessly, our solution aims to empower people with hearing loss to engage more fully in society and bolster security measures.

Key Words: Lip Reading, Machine Learning, Convolutional Neural Network (CNN)