Text: How do I load data from a csv file

[nawshad]

I have a dataset containing text and labels seperated by tabs. How I can load this dataset using torchtext?

[zhangguanheng66]

It's probably similar to the text classification datasets here.

@mttk Do you know if the current library support a csv file loading?

[bentrevett]

from torchtext import data

TEXT = data.Field()
LABEL = data.LabelField()

fields = [('text', TEXT), ('label', LABEL)]

train_data, test_data = data.TabularDataset.splits(
                            path = 'data',
                            train = 'train.csv',
                            test = 'test.csv',
                            format = 'tsv', #'tsv' for tabs, 'csv' for commas
                            fields = fields
)

[M-e-r-c-u-r-y]

@zhangguanheng66 Could we add the example @bentrevett posted in an example/usage section, torchtext doesn't really have any examples for external data sets. Adding a few examples for datasets that are not built into torchtext will help new users in understanding how to use torchtext better.

[zhangguanheng66]

We plan to eventually retire Field class as legacy code. However, at this moment, we could land a OSS PR as the example to help the usage case above. @M-e-r-c-u-r-y

[GaoJiqiang]

How can I load AG_news or DBpedia datasets from local csv file using 'text_classification.DATASETS' instead of from google drive?

[zhangguanheng66]

If you have the paths of train/test files for AG_NEWS and DBpedia, you could save them as a list and start from here. So why not just call the AG_NEWS and DBpedia API to load the datasets?

[GaoJiqiang]

If you have the paths of train/test files for AG_NEWS and DBpedia, you could save them as a list and start from here. So why not just call the AG_NEWS and DBpedia API to load the datasets?

Thx, because every time I have to use vpn to run the project for getting data from google drive, it's a little trouble. I want to download the CSV file and store them in local directory for convenience. I will try your method, best wishes.