Parlai: millions of personalized dialogue agents dataset inclusion?

[michaelshum]

will the dataset from Mazare et al's Training Millions of Personalized Dialogue Agents be included in the ParlAI tasks soon?

[stephenroller]

Due to distribution issues, I don't believe we'll be supporting this data set. In the future, we may enable a script to download the data, but it looks like this is out of the scope for the moment.

Closing, but please feel free to voice more concerns in this issue.

[taylorkangbeck]

I've been looking around for this dataset and haven't been able to find it.

@stephenroller To clarify, does that mean the data may be available elsewhere, just not in ParlAI? Or are we expected to manually reproduce the preprocessing steps in the paper on the original Reddit data?

[jaseweston]

@pemazare @samhumeau

[pemazare]

@taylorkangbeck Unfortunately, we won't be able to share the preprocessed data for legal reasons. However, the steps mentioned in the paper should allow you to reproduce the results.

[shatu]

@pemazare Are there any plans to release the preprocessing scripts for reproducibility purposes?

[stephenroller]

Probably not. They heavily depend on our internal infrastructure.