A Real-Time Lip Reading to Speech Android Application for Mute

Communication is one of the basic needs for every human being. However, for those suffering from muteness, communicating in their daily lives is a significant challenge. For those who are not aware of existing solutions like sign language or communication through typing, this paper proposes a Real Time Lip Reading to Speech Android Application that uses Artificial Intelligence for converting lip movement into speech. The proposed application uses Android’s CameraX for video recording, MediaPipe for lip region detection, and TensorFlow Lite for offline deep learning model prediction. The predicted words are then converted into speech using Android’s Text-toSpeech (TTS). The proposed application is designed for offline video upload processing without requiring internet connectivity. The proposed system’s effectiveness in assisting communication for those suffering from muteness has been tested in experiments.