LEVERAGING LARGE LANGUAGE MODELS AS EMBEDDING EXTRACTORS FOR IMPROVING ACCURACY AND BEYOND-ACCURACY PERFORMANCE IN COLLABORATIVE FILTERING RECOMMENDATION SYSTEMS

Recommendation systems play an essential role in empowering users by providing personalized suggestions tailored to their preferences and behaviors. Recently, the rise of Large Language Models (LLMs) has revolutionized the recommendation field, due to their remarkable capabilities in human language understanding and generation, along with providing highly contextualized representations for recommendation stakeholders by harnessing vast amounts of internet data. However, inherent societal biases often present in training data might reflect in LLMs being powerful yet potentially biased models. This study capitalizes on the growing interest in Large Language Models (LLMs) by leveraging a diverse array of pretrained embedding models to derive contextualized word representations from textual metadata of items within the MovieLens Latest small dataset. Obtained embeddings serve then as auxiliary information for the Collaborative Deep Learning (CDL) model [1]. CDL performances are comprehensively evaluated across varying cutoffs of recommendation lists, delving into both accuracy-focused metrics and beyond-accuracy dimensions such as fairness. Furthermore, tradeoffs between accuracy and fairness dimensions are quantitatively examined, providing insights into the nuanced interplay between these crucial aspects of recommendation systems design. Overall, the experimental outcomes underscore the inherent capabilities of CDL models to improve recommendation effectiveness while addressing fairness concerns, particularly evident at lower cutoff values, shedding light on the valuable potential of CDL in real-world settings with space constraints for recommended items e.g., mobile displays. Lastly, correlations between CDL and baseline models are researched in their performances, possibly implying shared mechanisms in the underlying algorithms. This analysis not only shows distinct clusters for CDL and baseline models, but also suggest that the choice between Bag of Words or LLMs as upstream source of word embeddings does not significantly influence the overall performance of the downstream CDL model, thereby highlighting the robustness of CDL models across different embedding sources.