Loading dataset from dataset/data.train.0.pt, number of examples: 93871
the process fails once i get to
[2019-03-05 12:34:01,506 INFO] Loading dataset from dataset/data.valid.0.pt, number of examples: 20000
and this is the error i get
Traceback (most recent call last):
File "train.py", line 109, in
main(opt)
File "train.py", line 39, in main
single_main(opt, 0)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/train_single.py", line 116, in main
valid_steps=opt.valid_steps)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/trainer.py", line 224, in train
valid_iter, moving_average=self.moving_average)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/trainer.py", line 274, in validate
outputs, attns = valid_model(src, tgt, src_lengths)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(input, *kwargs)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/models/model.py", line 42, in forward
enc_state, memory_bank, lengths = self.encoder(src, lengths)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(input, *kwargs)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/encoders/rnn_encoder.py", line 74, in forward
packed_emb = pack(emb, lengths_list)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/utils/rnn.py", line 148, in pack_padded_sequence
return PackedSequence(torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first))
RuntimeError: Length of all samples has to be greater than 0, but found an element in 'lengths' that is <= 0
should i define somewhere validation size? as it is much smaller than the regular size
I am using the last OpenNMT-py (git clomed today)
the process fails once it gets to validation
any way to define validation size without running into the error
thanks
you must have an empty line in the valid set.
I didn't find an empty line but i had /p that must have caused the error.
Thanks
Most helpful comment
you must have an empty line in the valid set.