A platform based on at-the-edge Intelligent Video Analytics for the rapid self-payment targeting commercial activity with table service

Starting date
January 1, 2022
Duration (months)
Computer Science, Department of Engineering for Innovation Medicine
Managers or local contacts
Bombieri Nicola

Joint Research 2021

The project aims at developing an AI-based platform for the rapid self-payment of customers targeting commercial activity with table service (i.e., restaurants). The platform is named D-PAY, and its purpose is to apply intelligent video analytics (IVA) to identify customers sit in the restaurant tables, track any customer who leaves the table and reaches the payment station to instantly provide the corresponding bill. The main constraint to satisfy when adopting IVA in public commercial activity is data privacy. To address actual GDPR rules, the implemented IVA applications have to be executed “at-the-edge”, whereby sensitive information kept by cameras are elaborated in real-time close to the input sensor. The project will start from existing inference applications based on convolutional neural networks (CNN) for the detection, tracking, and face recognition. It will combine such CNN models with post-processing, synchronization, and communication tasks to take advantage of programmable and distributed edge computing devices. To guarantee real-time computation on the resource constrained devices (i.e., NVIDIA Jetson), application tasks will customized and mapped on heterogeneous computing elements (i.e., CPU and GPU). Since the platform will have to be compliant to the ROS2 standard to guarantee modularity, portability, and scalability, and that ROS2 can lead to strong performance loss, the project will define efficient techniques to implement ROS2-compliant task communication. Finally, considering that the detection and tracking applications will be implemented by multiple distributed cameras with the corresponding computing devices, the project will compare the state-of-the-art communication standard to implement a protocol for secure real-time communication and synchronizations among edge devices. The platform will be validated in three commercial catering activities, each one with different architectural characteristics to evaluate applicability, accuracy, and scalability.

Project participants

Stefano Aldegheri
Research Scholarship Holders
Nicola Bombieri
Full Professor
Research areas involved in the project
Sistemi ciberfisici
Embedded and cyber-physical systems


Research facilities