The quest for reliable testing and verification in machine learning security

Abstract:
Machine Learning (ML) has revolutionized numerous domains, becoming the de facto standard for complex decision-making and automation. At the same time, the widespread integration of ML into safety-critical applications is introducing significant security concerns that must be identified and patched before deployment. However, unlike traditional software, ML models do not come with formal guarantees of correctness or robustness. Instead, they rely on patterns learned from data, which makes them inherently difficult to reason about and verify. Consequently, current evaluation approaches rely heavily on empirical testing, often without consistent standards.

This talk begins by introducing the foundations of ML security. We highlight key issues such as adversarial attacks that expose the fragility of current systems. We then shift focus to one of the central challenges in the field: evaluating the robustness of ML models remains largely an empirical process, with no widely accepted standards or formal verification methods in place. The seminar concludes by highlighting ongoing efforts to develop frameworks and tools that bring structure, reproducibility, and rigor to ML security evaluation.

Bio: Antonio Emanuele Cinà is an Assistant Professor (RTDA) at the University of Genoa, Italy. He received his Ph.D. (cum laude) in Computer Science at Ca' Foscari University of Venice in 2023, and he has been a Postdoc Researcher at the CISPA Helmholtz Center for Information Security, Saarbrücken, Germany. His research investigates security risks arising from spurious or adversarial correlations in artificial intelligence systems, which can cause unexpected behaviors (e.g., misclassification or the generation of harmful content), as well as robustness benchmarking and the development of verification tools for trustworthy ML. More recently, he has investigated the reliability of cybersecurity systems that integrate artificial intelligence solutions, aiming to understand their behavior for improving their accuracy, safety, and security.

link zoom: https://univr.zoom.us/j/88483412489