Recommenders: SAR single node adds columns to the train and test dataframes

Created on 6 Nov 2018  路  5Comments  路  Source: microsoft/recommenders

SAR single node implementation adds
'exponential' and 'rating_exponential' columns to the train data in fit(), and
'hashedUsers' column to the test data in recommend_k_items() or
hashedUsers' and 'hashedItems' columns in predict().

This would not be an expected behavior, especially when one uses/compares multiple models together by 're'-using the same set of train and test data to those models.

SAR pySpark implementation generates a copy of input dataframe and adds necessary columns to that when fit(), which we should do the same thing for the single node implementation.

bug needs discussion

All 5 comments

adding @maxkazmsft and @eisber for discussion

a) I agree that a copy feels more natural. If it's done for performance reasons, adding a pandas style inplace=True parameter is the way to go b) what's our investment into SAR single node?

Adding to Markus' point, if it's done for performance reasons, we can simply drop those added columns at the end of fitting or prediction so that the datasets include the original data again at the end.

@gramhagen do you know if this behavior is still happening?

no this has been fixed in staging by #387

Was this page helpful?
0 / 5 - 0 ratings

Related issues

miguelgfierro picture miguelgfierro  路  4Comments

karthikraja95 picture karthikraja95  路  3Comments

loomlike picture loomlike  路  5Comments

miguelgfierro picture miguelgfierro  路  6Comments

anargyri picture anargyri  路  4Comments