Opennmt-py: RuntimeError: Length of all samples has to be greater than 0 in validation set

Created on 5 Mar 2019  路  3Comments  路  Source: OpenNMT/OpenNMT-py

Loading dataset from dataset/data.train.0.pt, number of examples: 93871
the process fails once i get to
[2019-03-05 12:34:01,506 INFO] Loading dataset from dataset/data.valid.0.pt, number of examples: 20000
and this is the error i get

Traceback (most recent call last):
File "train.py", line 109, in
main(opt)
File "train.py", line 39, in main
single_main(opt, 0)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/train_single.py", line 116, in main
valid_steps=opt.valid_steps)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/trainer.py", line 224, in train
valid_iter, moving_average=self.moving_average)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/trainer.py", line 274, in validate
outputs, attns = valid_model(src, tgt, src_lengths)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(input, *kwargs)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/models/model.py", line 42, in forward
enc_state, memory_bank, lengths = self.encoder(src, lengths)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(input, *kwargs)
File "/usr/local/lib/python3.5/dist-packages/OpenNMT2-py/OpenNMT-py/onmt/encoders/rnn_encoder.py", line 74, in forward
packed_emb = pack(emb, lengths_list)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/utils/rnn.py", line 148, in pack_padded_sequence
return PackedSequence(torch._C._VariableFunctions._pack_padded_sequence(input, lengths, batch_first))
RuntimeError: Length of all samples has to be greater than 0, but found an element in 'lengths' that is <= 0

should i define somewhere validation size? as it is much smaller than the regular size

Most helpful comment

you must have an empty line in the valid set.

All 3 comments

I am using the last OpenNMT-py (git clomed today)
the process fails once it gets to validation
any way to define validation size without running into the error
thanks

you must have an empty line in the valid set.

I didn't find an empty line but i had /p that must have caused the error.
Thanks

Was this page helpful?
0 / 5 - 0 ratings

Related issues

memray picture memray  路  4Comments

KelleyYin picture KelleyYin  路  6Comments

andreaferretti picture andreaferretti  路  5Comments

azureskyL picture azureskyL  路  3Comments

nikhilweee picture nikhilweee  路  3Comments