Datasets: [Dataset requests] New datasets for Text Classification

[thomwolf]

We are missing a few datasets for Text Classification which is an important field.

Namely, it would be really nice to add:

• TREC-6 dataset (see here for instance: https://pytorchnlp.readthedocs.io/en/latest/source/torchnlp.datasets.html#torchnlp.datasets.trec_dataset) [done]
• Yelp-5
• Movie review (Movie Review (MR) dataset [156]) [done (same as rotten_tomatoes)]
• SST (Stanford Sentiment Treebank) [include in glue]
• Multi-Perspective Question Answering (MPQA) dataset [require authentication (indeed manual download)]
• Amazon. This is a popular corpus of product reviews collected from the Amazon website [159]. It contains labels for both binary classification and multi-class (5-class) classification
• 20 Newsgroups. The 20 Newsgroups dataset [done]
• Sogou News dataset [done]
• Reuters news. The Reuters-21578 dataset [165] [done]
• DBpedia. The DBpedia dataset [170]
• Ohsumed. The Ohsumed collection [171] is a subset of the MEDLINE database
• EUR-Lex. The EUR-Lex dataset
• WOS. The Web Of Science (WOS) dataset [done]
• PubMed. PubMed [173]
• TREC-QA. TREC-QA
• Quora. The Quora dataset [180]

All these datasets are cited in https://arxiv.org/abs/2004.03705

[thomwolf]

Pinging @mariamabarham as well

[jxmorris12]

• nlp has MR! It's called rotten_tomatoes
• SST is part of GLUE, or is that just SST-2?
• nlp also has ag_news, a popular news classification dataset

I'd also like to see:

• the Yahoo Answers topic classification dataset
• the Kaggle Fake News classification dataset

[mariamabarham]

Thanks @jxmorris12 for pointing this out.

In glue we only have SST-2 maybe we can add separately SST-1.

[jxmorris12]

This is the homepage for the Amazon dataset: https://www.kaggle.com/datafiniti/consumer-reviews-of-amazon-products

Is there an easy way to download kaggle datasets programmatically? If so, I can add this one!

[mariamabarham]

Hi @jxmorris12 for now I think our dl_manager does not download from Kaggle.
@thomwolf , @lhoestq

[ghomasHudson]

Pretty sure the quora dataset is the same one I implemented here: https://github.com/huggingface/nlp/pull/366

[moscow25]

Great list. Any idea if Amazon Reviews has been added?

• ~40 GB of text (sadly no emoji)
• popular MLM pre-training dataset before bigger datasets like WebText https://arxiv.org/abs/1808.01371
• turns out that binarizing the 1-5 star rating leads to great Pos/Neg/Neutral dataset, T5 paper claims to get very high accuracy (98%!) on this with small amount of finetuning https://arxiv.org/abs/2004.14546

Apologies if it's been included (great to see where) and if not, it's one of the better medium/large NLP dataset for semi-supervised learning, albeit a bit out of date.

Thanks!!

cc @sshleifer

[moscow25]

On the Amazon Reviews dataset, the original UCSD website has noted these are now updated to include product reviews through 2018 -- actually quite recent compared to many other datasets. Almost certainly the largest NLP dataset out there with labels!
https://jmcauley.ucsd.edu/data/amazon/

Any chance someone has time to onboard this dataset in a HF way?

cc @sshleifer