MACHINE UNLEARNING AND LOW-COST MODEL EDITING FOR MATRIX FACTORIZATION-BASED RECOMMENDER SYSTEMS

Abstract

Collaborative filtering (CF) models typically undertake various tasks in large-scale recommender systems (RecSys), such as providing low-rank embedding features or generating candidate lists, making them an indispensable component of modern multistage RecSys. Unfortunately, due to the nature of CF models modelling bipartite interaction graphs, they lack flexibility in responding to user feedback, necessitating extensive recalculations when new interaction data is received. In real-world RecSys, data and user preferences change in real-time, further exacerbating the computational burden and response delay when CF models handle dynamic updates. To address this challenge, during my master’s program, I explored two research directions: unlearning in RecSys and efficient model editing methods. First, we used simple formulas to adjust candidate lists without modifying the model for real-time user data unlearning requests. We proposed the Forgetting Stability Boundary (FSB) as the feasibility threshold for unlearning in RecSys. FSB is a new metric for assessing the efficiency of machine unlearning in item-based recommendation systems, determining when it is more resource-efficient than traditional retraining. By analyzing various factors affecting RecSys machine unlearning behaviour, we found that the embedding dimension and cumulative interaction rate (CIR) significantly impact unlearning performance. In FSB, simple formulas can unlearn user data without modifying the model. Second, we proposed an efficient model editing method for matrix factorization (MF)-based collaborative filtering models, called Model Editing for Collaborative Filtering (ME-CF), which reduces retraining costs through linear approximation. The proposed method can approximate the latest user and item embeddings through linear operations on the previously estimated MF model without retraining by fitting the model with a small amount of data. ME-CF can accurately update embeddings by solving quadratic equations, significantly reducing computational costs and update time. Our empirical evaluation of the proposed methods on three well-known benchmark datasets (such as Movielens and Amazon Software) demonstrates their significant advantages over industry-standard full-model retraining methods. ME-CF, in particular, substantially reduces computational costs and time while maintaining high-quality embeddings. This makes it particularly suitable for rapidly changing online recommendation services. This thesis posits that the proposed methods are both feasible and effective in enhancing the adaptability and efficiency of collaborative filtering models in large-scale recommender systems, offering new solutions for practical applications with broad potential applicability.