Flex Sensor and Machine Learning-Based Mute Communication System

Flex Sensor and Machine Learning-Based Mute Communication System

Srijani Ghosh1, Darshan Giri2, Vinamratha M.N3, Nandini M4, Radha R.C5, Latha H.N6

1Srijani Ghosh, Electronics and Communication, BMSCE

2Darshan Giri, Electronics and Communication, BMSCE

3Vinamratha M.N., Electronics and Communication, BMSCE

 4Nandini M, Electronics and Communication, BMSCE

 5Radha R.C., Electronics and Communication, BMSCE

6Latha H.N., Electronics and Communication, BMSCE

---------------------------------------------------------------------***---------------------------------------------------------------------

Abstract

Individuals facing speech impairments encounter significant communication challenges. To address this, we have developed a groundbreaking system that seamlessly translates custom hand gestures into written text and spoken language. Our system combines flex sensors and state-of-the-art machine learning techniques, empowering individuals with speech impairments to express themselves effectively and improve their overall communication experience. By connecting flex sensors to an Arduino board and utilizing MATLAB for robust data collection, analysis, and processing, we achieve a highly accurate gesture recognition system. Through meticulous steps of data labeling, rigorous training of a machine learning model, and thorough accuracy evaluations, our system recognizes a wide range of gestures, including numbers 0-9, letters A-Z, and 10 Kannada language gestures conveying meaningful phrases. The hand gesture recognition system achieved notable performance with an accuracy of 83.5%, precision of 88.6%, recall of 95.1%, and F1 Score of 91.7%.

Key Words:   speech impairments, hand gestures, gesture recognition, flex sensors, machine learning, communication accessibility