Text: BucketIterator goes into infinite loop

Created on 13 Feb 2018  路  1Comment  路  Source: pytorch/text

When I use the bucketIterator with a tabulardataset object, it is going into infinite loop in training.
Here is the example of the use:
train = data.TabularDataset(
path= review_dir + d_review_dir + 'train.tsv', format='tsv', skip_header=True,
fields=[('text', TEXT),
('labels', LABELS)])

train_iter, dev_iter, test_iter = data.Iterator.splits(
                            (train, val, test), device = -1,
                            batch_sizes=(args.batch_size, len(val), len(test)))

When I loop through the train_iter object with: for batch in train_iter:
it goes into an infinite loop, please fix the error.

Most helpful comment

If you do not want the iterator to go into an infinite loop, you need to pass the keyword argument repeat=False like

train_iter, dev_iter, test_iter = data.Iterator.splits(
                            (train, val, test), device = -1,
                            repeat=False,
                            batch_sizes=(args.batch_size, len(val), len(test)))

>All comments

If you do not want the iterator to go into an infinite loop, you need to pass the keyword argument repeat=False like

train_iter, dev_iter, test_iter = data.Iterator.splits(
                            (train, val, test), device = -1,
                            repeat=False,
                            batch_sizes=(args.batch_size, len(val), len(test)))
Was this page helpful?
0 / 5 - 0 ratings