SEQUENCE-LEVEL PREDICTORS OF RECOMMENDATION EFFECTIVENESS

Sequential recommendation is a central task in modern recommender systems, where models generate ranked lists of items from users’ temporally ordered interaction histories. Although these models often achieve strong aggregate performance, their effectiveness can vary substantially across individual user sequences. In some cases, the produced ranking may therefore be poorly aligned with the user’s current intent. This work studies performance prediction for sequential recommendation. Inspired by Query Performance Prediction (QPP) in document retrieval, an investigation is conducted to determine whether the effectiveness of a recommendation list can be estimated before observing the user’s future interactions. The task is first formalised at the level of individual user sequences, where the goal is to predict the expected quality of a ranked list without access to ground-truth future items. Existing QPP predictors are then adapted to the sequential recommendation setting, and additional predictors tailored to user sequences, model scores, item embeddings, and ranked-list properties are introduced. The proposed predictors are organised into several signal-based families, including user sequence characteristics, score distributions, result coherence, ranking robustness, novelty, and diversity. These predictors are evaluated on standard benchmark datasets and multiple sequential recommendation models by measuring their correlation with observed per-user recommendation effectiveness. The findings indicate that performance predictors indeed correlate meaningfully with metrics in sequential recommendation tasks across different models and datasets. These findings suggest the potential for adaptive recommendation strategies that respond to predicted low recommendation quality.