A Real-Time Lip Reading to Speech Android Application for Mute
Puppala Ramya Sri , Dr.Mogili Ravinder
Assistant Professor, Professor and Head
Dept of Computer Science and Engineering (AIML)
Jyothishmathi Institute of Technology and Science (JNTUH)
Karimnagar, Telangana, India ramyasripuppala1244@gmail.com
Karimnagar, Telangana, India m mogili.ravinder@jits.ac.in
Cheekati Abhinay
UG Student
Dept. of Computer Science and Engineering
Jyothishmathi Institute of Technology and Science (JNTUH)
Karimnagar, Telangana, India
cheekatiabhinay63@gmail.com
Nagula Deekshitha
UG Student
Dept. of Computer Science and Engineering
Jyothishmathi Institute of Technology and Science (JNTUH) Karimnagar, Telangana, India
naguladeekshitha@gmail.com
Gourishetty Pavani
UG Student
Dept. of Computer Science and Engineering
Jyothishmathi Institute of Technology and Science (JNTUH) Karimnagar, Telangana, India
gourishettypavani@gmail.com
Podeti Ajith
UG Student
Dept. of Computer Science and Engineering
Jyothishmathi Institute of Technology and Science (JNTUH)
Karimnagar, Telangana, India
podetiajith@gmail.com
Abstract—Communication is one of the basic needs for every human being. However, for those suffering from muteness, communicating in their daily lives is a significant challenge. For those who are not aware of existing solutions like sign language or communication through typing, this paper proposes a Real Time Lip Reading to Speech Android Application that uses Artificial Intelligence for converting lip movement into speech. The proposed application uses Android’s CameraX for video recording, MediaPipe for lip region detection, and TensorFlow Lite for offline deep learning model prediction. The predicted words are then converted into speech using Android’s Text-toSpeech (TTS). The proposed application is designed for offline video upload processing without requiring internet connectivity. The proposed system’s effectiveness in assisting communication for those suffering from muteness has been tested in experiments.
Index Terms—Lip Reading, Visual Speech Recognition, Computer Vision, TensorFlow Lite, MediaPipe, Offline AI, Android Application, Assistive Technology
