Pubblicazioni

Autori:

Tur, Anil; Conti, Alessandro; Beyan, Cigdem; Boscaini, Davide; Larcher, Roberto; Messelodi, Stefano; Poiesi, Fabio; Ricci, Elisa

Titolo:

Exploring Fine-grained Retail Product Discrimination with Zero-shot Object Classification Using Vision-Language Models

Anno:

2024

Tipologia prodotto:

Contributo in atti di convegno

Tipologia ANVUR:

Contributo in Atti di convegno

Lingua:

Inglese

Titolo del Convegno:

IEEE 8th Forum on Research and Technologies for Society and Industry Innovation (RTSI)

Luogo:

Lecco

Periodo:

september 18-20, 2024

Intervallo pagine:

1-6

Parole chiave:

zero-shot, object classification, smart retail systems, vision-language models, clip

Breve descrizione dei contenuti:

In smart retail applications, the large number of products and their frequent turnover necessitate reliable zero-shot object classification methods. The zero-shot assumption is essential to avoid the need for re-training the classifier every time a new product is introduced into stock or an existing product undergoes rebranding. In this paper, we make three key contributions. Firstly, we introduce the MIMEX dataset, comprising 28 distinct product categories. Unlike existing datasets in the literature, MIMEX focuses on fine-grained product classification and includes a diverse range of retail products. Secondly, we benchmark the zero-shot object classification performance of state-of-the-art vision-language models (VLMs) on the proposed MIMEX dataset. Our experiments reveal that these models achieve unsatisfactory fine-grained classification performance, highlighting the need for specialized approaches. Lastly, we propose a novel ensemble approach that integrates embeddings from CLIP and DINOv2 with dimensionality reduction techniques to enhance classification performance. By combining these components, our ensemble approach outperforms VLMs, effectively capturing visual cues crucial for fine-grained product discrimination. Additionally, we introduce a class adaptation method that utilizes visual prototyping with limited samples in scenarios with scarce labeled data, addressing a critical need in retail environments where product variety frequently changes. To encourage further research into zero-shot object classification for smart retail applications, we will release both the MIMEX dataset and benchmark to the research community. Interested researchers can contact the authors for details on the terms and conditions of use. The code is available: https://github.com/AnilOsmanTur/Zero-shot-Retail-Product-Classification.

Id prodotto:

140669

Handle IRIS:

11562/1132652

ultima modifica:

1 agosto 2024

Citazione bibliografica:

Tur, Anil; Conti, Alessandro; Beyan, Cigdem; Boscaini, Davide; Larcher, Roberto; Messelodi, Stefano; Poiesi, Fabio; Ricci, Elisa, Exploring Fine-grained Retail Product Discrimination with Zero-shot Object Classification Using Vision-Language Models  in Proceedings of 2024 IEEE 8th Forum on Research and Technologies for Society and Industry Innovation (RTSI) , Atti di "IEEE 8th Forum on Research and Technologies for Society and Industry Innovation (RTSI)" , Lecco , september 18-20, 2024 , 2024 , pp. 1-6

Consulta la scheda completa presente nel repository istituzionale della Ricerca di Ateneo