AutoWatcher: a Real-Time Context-Aware Security Alert System using LLMs

AutoWatcher: a Real-Time Context-Aware Security Alert System using LLMs

Praneeth Vadlapati
University of Arizona,
Tucson, USA
praneethv@arizona.edu
ORCID: 0009-0006-2592-2564

Abstract: Ensuring the safety of every person is a necessity. Burglaries are frequent across residential and commercial spaces. Criminals frequently use weapons and are a serious threat to the common people who are innocent. The best solution to safety is alertness. Computer Vision (CV) is frequently implemented through neural networks that use a process of training to gain abilities to detect humans. Multimodal Large Language Models (LLMs) capable of vision-based tasks are common and are capable of processing images similar to humans who use their analytical abilities. With the ongoing research, LLMs are becoming smaller, faster, more accurate, and cost-effective, making them usable in security systems to detect threats in an intelligent way based on custom instructions. Existing research does not explore the usage of multimodal LLMs for real-time monitoring. This paper proposes a system called “AutoWatcher” to monitor a place using a camera in real-time, constantly detect humans, and use LLMs to assess potential threats. The system is designed for two levels of alerts, one when a person is detected and another when the LLM declares them as suspicious. This ensures alertness of the people who can proactively protect their lives and valuable assets on time before a security breach occurs at their home or workspace. This enables the residents to alert the emergency service authorities in their locality. The system has successfully detected suspicious people with an accuracy of up to 90%. The accuracy is 100% in some test cases. The code is available at github.com/Pro-GenAI/AutoWatcher.

Keywords: large language models (LLMs), computer vision (CV), multimodal LLMs, neural networks, edge AI, automated surveillance