Recommenders: [BUG] Review all recommend_top_k algos

[miguelgfierro]

Description

There are some inconsistencies between the implementations of recommend_top_k. In SAR we are using the users of the test set whereas in SVD we are using the users in the train set. Sometimes they can be the same, but not in general.

Also, as suggested by @yueguoguo, there is a function to optimally remove the seen items, but each algo use its own function

[gramhagen]

this is a good catch, pretty much all the approaches can work the same if set to score the set of user-item pairs (generated from the crossjoin of unique users and items in the test set, with seen items removed). I think we do that once (using the method @yueguoguo mentioned) in the benchmark script, and then adjust the recommend_k_x methods to call the algo's score method then rank the results.

Actually, can we use the same results from the rating stage (assuming we're generating the same test set described above) and just rank them?

[miguelgfierro]

I think we do that once (using the method @yueguoguo mentioned) in the benchmark script, and then adjust the recommend_k_x methods to call the algo's score method then rank the results.

agree

[miguelgfierro]

We need to take a look at the fastai function when we are removing the seen items. This can be improved

        total_users, total_items = model.data.train_ds.x.classes.values()
        total_items = total_items[1:]
        total_users = total_users[1:]
        test_users = test[DEFAULT_USER_COL].unique()
        test_users = np.intersect1d(test_users, total_users)
        users_items = cartesian_product(test_users, total_items)
        users_items = pd.DataFrame(users_items, columns=[DEFAULT_USER_COL, DEFAULT_ITEM_COL])
        training_removed = pd.merge(users_items, train.astype(str), on=[DEFAULT_USER_COL, DEFAULT_ITEM_COL], how='left')
        training_removed = training_removed[training_removed[DEFAULT_RATING_COL].isna()][[DEFAULT_USER_COL, DEFAULT_ITEM_COL]]

[anargyri]

A better practice (from an ML perspective) would be to remove the training data and the remove_seen flag from the recommend_k() methods and input the set of available items instead. This set would need to be computed outside the method.
A further option is to input users and items instead of a single dataframe. That would be more convenient to the user if they want to do something other than "remove seen" or "include_seen".

[gramhagen]

If recommend_k operates only on input users then retaining remove seen as an option makes sense. This may only apply to algos which maintain the full set of items internally.

Pregenerating the user-item pairs is fine but should go through a score method not recommend_k.

[gramhagen]

yeah, maybe a quick benchmark on which is the fastest way to generate all user-item pairs df. then update dataset.pandas_df_utils.user_item_pairs. we can also leverage common.python_utils.get_top_k_scored_items to speed up getting recommended items.

[anargyri]

If recommend_k operates only on input users then retaining remove seen as an option makes sense. This may only apply to algos which maintain the full set of items internally.

Pregenerating the user-item pairs is fine but should go through a score method not recommend_k.

My point was not about implementation, but about the interface we expose to the data scientist. I am proposing that this is very simple i.e. something like
recommend_k(set of users, set of items)
and recommend_k() outputs the top-k items among set_of_items per user.

[miguelgfierro]

I like the approach of @anargyri, that way we are making the code more modular. Are you thinking of something like this:

df = remove_seen_items(df_original)
preds = model.recommend_k_items(df)

[anargyri]

I like the approach of @anargyri, that way we are making the code more modular. Are you thinking of something like this:

df = remove_seen_items(df_original)
preds = model.recommend_k_items(df)

More like
unseen_items = remove_seen_items(df_original) users = df_original[usercol].unique() preds = model.recommend_k_items(users, unseen_items)

The idea is to be able to cover other situations as well. For example, the data scientist may want to recommend only to a subset of users (e.g. the "warmer" ones). Or they may want to exclude certain items (e.g. not show Chinese movies to users from France).

[gramhagen]

i'm confused, aren't the unseen items only unseen for a specific user? this sounds like excluding sets of items across all users.

I was thinking that you may only want to recommend novel items for a user, in which case you would need to know which item-user pairs were in the training set and remove them from the user-item pairs of the test set

[anargyri]

i'm confused, aren't the unseen items only unseen for a specific user? this sounds like excluding sets of items across all users.

I was thinking that you may only want to recommend novel items for a user, in which case you would need to know which item-user pairs were in the training set and remove them from the user-item pairs of the test set

Yes, you are right. I am thinking a bit more generally than the case of seen-unseen items, for example, remove Chinese movies for French users and French movies for Chinese users. I guess Miguel was also thinking of the same thing. Maybe my syntax is a bit too complicated because the items would be a dictionary of lists. Miguel's syntax with dataframes is probably better.

[yueguoguo]

I think it might be more user friendly to have the "list of the users" as input to function recommend_k_items. In this way, it might be also useful to enhance the computation efficiency, as the user list can be used for subsetting the recommended user-(k)-item pairs. It is also easy to maintain the list of users instead of a dataframe that contains user, item, and probably rating columns.

Currently we have the basic functions like cross join of user-item pairs, with which we can wrap up for performing the "removing seen" operations. It takes dataframes as input however, so we can probably think of implementing something new?

[miguelgfierro]

I think we should consider using a data frame, with the same columns as in the other functions, users, items...
It is more consistent with the rest of the code and hopefully it's easier to understand by the user

[miguelgfierro]

I think we should consider using a data frame, with the same columns as in the other functions, users, items...
It is more consistent with the rest of the code and hopefully it's easier to understand by the user