Opennmt-py: torchtext 0.2 issue to check

[laubonghaudoi]

I trained the model successfully and run the command python translate.py -model demo-model_epochX_PPL.pt -src data/src-test.txt -output pred.txt -replace_unk -verbose in README to translate, but it occurred the following error:

WARNING: -batch_size isn't supported currently, we set it to 1 for now!
Loading model parameters.
Traceback (most recent call last):
  File "translate.py", line 127, in <module>
    main()
  File "translate.py", line 66, in main
    for batch in test_data:
  File "/home/yzf/anaconda3/lib/python3.6/site-packages/torchtext/data/iterator.py", line 178, in __iter__
    self.train)
  File "/home/yzf/anaconda3/lib/python3.6/site-packages/torchtext/data/batch.py", line 22, in __init__
    setattr(self, name, field.process(batch, device=device, train=train))
  File "/home/yzf/anaconda3/lib/python3.6/site-packages/torchtext/data/field.py", line 184, in process
    tensor = self.numericalize(padded, device=device, train=train)
  File "/home/yzf/anaconda3/lib/python3.6/site-packages/torchtext/data/field.py", line 294, in numericalize
    arr = [numericalization_func(x) for x in arr]
  File "/home/yzf/anaconda3/lib/python3.6/site-packages/torchtext/data/field.py", line 294, in <listcomp>
    arr = [numericalization_func(x) for x in arr]
TypeError: float() argument must be a string or a number, not 'torch.LongTensor'

It seems that this issue is a duplicate of #352, but I run this in python 3.6.

[JianyuZhan]

Hi, @jekbradbury, would you please help look at this? I am not sure if torchtext 0.2.0 breaks this. The numericalization_func is fetched from Field.tensor_types map, in this case it is float(), thus caused this error.

[laubonghaudoi]

I reinstalled torchtext 0.1.1 and the codes work now. I agree with @thammegowda in #345 that the requirements.txt should force the torchtext==0.1.1

[jekbradbury]

What are the properties of the Field that causes this issue? (as in, what are the values of field.sequential, field.use_vocab, and field.__class__)? Also, does the patch in https://github.com/pytorch/text/pull/159 fix this problem for you?

[zphang]

Tried the fix in pytorch/text/pull/159, but ran into an issue in the next couple lines:

  File "/home/zphang/miniconda3/envs/nlpclass/lib/python3.6/site-packages/torchtext-0.2.0-py3.6.egg/torchtext/data/iterator.py", line 178, in __iter__
  File "/home/zphang/miniconda3/envs/nlpclass/lib/python3.6/site-packages/torchtext-0.2.0-py3.6.egg/torchtext/data/batch.py", line 22, in __init__
  File "/home/zphang/miniconda3/envs/nlpclass/lib/python3.6/site-packages/torchtext-0.2.0-py3.6.egg/torchtext/data/field.py", line 184, in process
  File "/home/zphang/miniconda3/envs/nlpclass/lib/python3.6/site-packages/torchtext-0.2.0-py3.6.egg/torchtext/data/field.py", line 296, in numericalize
TypeError: make_src() takes 2 positional arguments but 3 were given

Seems like make_src takes 2 arguments, but it is given 3 in https://github.com/pytorch/text/blob/master/torchtext/data/field.py#L296