SAR single node implementation adds
'exponential' and 'rating_exponential' columns to the train data in fit(), and
'hashedUsers' column to the test data in recommend_k_items() or
hashedUsers' and 'hashedItems' columns in predict().
This would not be an expected behavior, especially when one uses/compares multiple models together by 're'-using the same set of train and test data to those models.
SAR pySpark implementation generates a copy of input dataframe and adds necessary columns to that when fit(), which we should do the same thing for the single node implementation.
adding @maxkazmsft and @eisber for discussion
a) I agree that a copy feels more natural. If it's done for performance reasons, adding a pandas style inplace=True parameter is the way to go b) what's our investment into SAR single node?
Adding to Markus' point, if it's done for performance reasons, we can simply drop those added columns at the end of fitting or prediction so that the datasets include the original data again at the end.
@gramhagen do you know if this behavior is still happening?
no this has been fixed in staging by #387