Investigating the Correlation Between Training Data Influence and Memorization in AI Models

Abstract

This thesis explores the relationship between influence and memorization of training data in deep learning models. Influence metrics aim to quantify the impact of individual training examples on model predictions, while memorization scores assess how much specific data points are retained by the model. Understanding the connection between these two concepts is important for both interpretability and privacy in AI systems. Prior research has shown that memorized training data might be leaked when the corresponding trained model makes inferences. This work examines whether influence metrics can serve as indicators of memorization and investigates how this relationship varies across different neural network architectures. By exploring whether training data items with higher influence also demonstrate higher memorization, the study aims to identify data points that may be more prone to leakage, thereby contributing to more privacy-aware AI systems.