Virtual Mouse & Voice Assistance

Virtual Mouse & Voice Assistance

Akshay Godhale, Ayush Warang, Vedant Kajrekar, Rohit Barve

Abstract— Artificial intelligence technology has paved the way for hand gesture recognition in controlling virtual devices. This paper introduces a system that uses AI to identify hand gestures and convert them into movements of a virtual mouse, providing an alternative interface for those who find traditional mice or keyboards challenging. The system captures images of the user’s hand using a camera, which are then processed by an AI algorithm to identify the gestures. These gestures are translated into corresponding mouse actions on a virtual screen. The system is adaptable and scalable, capable of being controlled virtually using dynamic/static hand gestures and a voice assistant. It utilizes Machine Learning and Computer Vision algorithms without any additional hardware requirements. The model is implemented using a Convolutional Neural Network and the mediapipe framework. This system could potentially enable hands-free operation of devices in hazardous environments and provide an alternative to hardware mice, thereby enhancing user experience and accessibility

Keywords— Computer vision, hand gesture recognition, Media-pipe, voice assistant & virtual mouse.