Various notebooks
For all notebooks that use Movielens and all algorithms we should compute metrics only on items by the user (i.e. we do not recommend already seen movies again).
Remember to edit tests.
Hi @anargyri, let me see if I understood the issue correctly: currently (top k) prediction is done on the test set; in principle, the prediction can contain seen movies, i.e. movies that are in the train set. As the scores are evaluated by looking at the movies in the test set alone, including seen (train set) movies can alter this metric in case the latter how up in top k. Is this correct?
@WessZumino Correct. Removing seen items or not depend on the actual business scenario. In the case of Movie recommendation, the assumption is that we don't want to recommend movies that a user has watched before. Thus the seen movies should be removed.
@WessZumino we support both options, so the user can choose according to their use case. This issue is about making all notebooks running on Movielens data consistent i.e. we remove seen items in all cases. It was already like this whenever we call SAR, which has an option remove_seen.