A Multi-Agent Real-Time Surveillance System for Object Abandonment, Crowd Anomaly Detection, and Face Recognition

Abstract

This project presents the development of a real-time, multi-agent surveillance system designed to detect abandoned luggage, monitor crowd density, and perform face recognition for access control. The system leverages deep learning models, YOLOv5 for object detection and FaceNet for facial identification, integrated within a lightweight Python-based framework supporting parallel camera agents. A dual-agent architecture, consisting of wide-view and zoom-focused modules, enables distributed scene monitoring and targeted inspection. The system achieves real-time performance through CUDA acceleration and minimal preprocessing, making it suitable for live deployment scenarios.

Evaluation in a controlled indoor environment demonstrated high person detection accuracy, adaptable anomaly detection based on historical density, and reliable face recognition with moderate false positive rates. Alerts are dispatched via Pushbullet, with future integration planned for FIWARE to enable smart city compatibility. While promising, the system’s performance in complex public environments remains to be validated. Future work will focus on improving detection accuracy, expanding to multi-modal inputs, and conducting real-world testing across high-traffic surveillance environments.