Opennmt-py: Pytorch 0.4 support: python2/3 issue with detach()

[David-Levinthal]

This occurs on systems with either cuda8/cudnn6 or cuda9/cudnn7, Ubuntu 16.04. Pytorch built from source.

python3 train.py -data data/demo -save_model demo-model -gpuid 1

    main()
  File "train.py", line 299, in main
    train_model(model, train, valid, fields, optim)
  File "train.py", line 159, in train_model
    train_stats = trainer.train(epoch, report_func)
  File "/home/levinth/OpenNMT-py/onmt/Trainer.py", line 133, in train
    dec_state.detach()
  File "/home/levinth/OpenNMT-py/onmt/Models.py", line 444, in detach
    h.detach_()
RuntimeError: Can't detach views in-place. Use detach() instead

[David-Levinthal]

This issue occurs on python2.7 as well. The root may lie in pytorch softmax?

/home/levinth/OpenNMT-py/onmt/modules/GlobalAttention.py:150: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  align_vectors = self.sm(align.view(batch*targetL, sourceL))
/usr/local/lib/python2.7/dist-packages/torch/nn/modules/container.py:72: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
  input = module(input)
Traceback (most recent call last):
  File "train.py", line 303, in <module>
    main()
  File "train.py", line 299, in main
    train_model(model, train, valid, fields, optim)
  File "train.py", line 159, in train_model
    train_stats = trainer.train(epoch, report_func)
  File "/home/levinth/OpenNMT-py/onmt/Trainer.py", line 133, in train
    dec_state.detach()
  File "/home/levinth/OpenNMT-py/onmt/Models.py", line 444, in detach
    h.detach_()
RuntimeError: Can't detach views in-place. Use detach() instead

[JianyuZhan]

Can you report the pytorch version? Have you tried a older version of pytorch on PyPI, I didn't encounter this on my test, I am afraid what has been changed in latest pytorch breaks this.

[David-Levinthal]

That was what I suspected :-)
this is a top of tree build of pytorch from source
the version string check returns
python
Python 2.7.12 (default, Nov 20 2017, 18:23:56)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import torch
print(torch.__version__)
0.4.0a0+80c8635

d

On Sat, Dec 2, 2017 at 7:56 PM, Jianyu Zhan [e-mail]
wrote:

Can you report the pytorch version? Have you tried a older version of
pytorch on PyPI, I didn't encounter this on my test, I am afraid what has
been changed in latest pytorch breaks this.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenNMT/OpenNMT-py/issues/403#issuecomment-348738488,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIUuT99C11iN-BJwIi1IUFVYC9xYccM1ks5s8hvQgaJpZM4QyorZ
.

[David-Levinthal]

note I need the latest version to get good support for cuda9/cudnn7/nccl2
I have V100s
d

On Sun, Dec 3, 2017 at 10:30 AM, David Levinthal [email protected]

wrote:

That was what I suspected :-)
this is a top of tree build of pytorch from source
the version string check returns
python
Python 2.7.12 (default, Nov 20 2017, 18:23:56)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import torch
print(torch.__version__)
0.4.0a0+80c8635

d

On Sat, Dec 2, 2017 at 7:56 PM, Jianyu Zhan [e-mail]
wrote:

Can you report the pytorch version? Have you tried a older version of
pytorch on PyPI, I didn't encounter this on my test, I am afraid what has
been changed in latest pytorch breaks this.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenNMT/OpenNMT-py/issues/403#issuecomment-348738488,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIUuT99C11iN-BJwIi1IUFVYC9xYccM1ks5s8hvQgaJpZM4QyorZ
.

[David-Levinthal]

you mean changing
for h in self._all:
if h is not None:
h.detach_()
to
for h in self._all:
if h is not None:
h = h.detach_()

in models.py?
d

On Mon, Dec 4, 2017 at 12:11 AM, Jianyu Zhan [e-mail]
wrote:

Will h = h.detach() work for you? If not, I will need to delve into the
pytorch to figure out what is wrong.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenNMT/OpenNMT-py/issues/403#issuecomment-348888960,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIUuT0vwcC4SErYRhyl2wjrRDFOL6thMks5s86lJgaJpZM4QyorZ
.

[maksymbevza]

I think h = h.detach() will not solve the issue, because self._all will still point to the old Tensor/Variable that is not detached. Am I right?

[playma]

I face the same probelm in Pytorch 0.4.0 in Python 3.5.2.
I thought that the key is the pytorch version not python version, because there is no problem when I use Pytorch 0.2.0 in Python 3.5.2

[playma]

There is the similar discussion here
https://github.com/pytorch/pytorch/issues/3674

I don't know is it useful.

[srush]

This is helpful thanks. We haven't yet tested with the newest versions of pytorch. We will add these to our travis and update.

[David-Levinthal]

Tried again just now using python3 and new download of OpenNMT. Same issue
OpenNMT-py/onmt/Models.py", line 572, in detach
h.detach_()
RuntimeError: Can't detach views in-place. Use detach() instead

[srush]

Can you try v0.3 of pytorch. We will try to update to v0.4 soon.

[Halfish]

As suggested, I changed h.detach_() to h.detach(), and it works fine for me.

The version of my pytorch is 0.4.

[David-Levinthal]

thanks!

On Mon, Feb 26, 2018 at 6:52 AM, Bruce [e-mail] wrote:

As suggested, I changed h.detach_() to h.detach(), and it works fine for
me. The version of My pytorch is 0.4~

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenNMT/OpenNMT-py/issues/403#issuecomment-368528138,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIUuTzGlah3uq4d8mzmwxyIyQn3_TAjNks5tYsVJgaJpZM4QyorZ
.

[patelrajnath]

@Halfish
After changing h.detach_() to h.detach(), I'm getting the following error-
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

[da03]

In onmt/Loss.py, change loss.backward() to loss.backward(retain_graph=True) might fix the issue.

[patelrajnath]

yes, I tried that (resolves the error) but at the time of training, perplexity doesn't change at all ie. it doesn't train actually.

[da03]

Hmm weird, I was able to train it though. Is it possible that there's something else that you broke?

[patelrajnath]

I only changed above two mentioned modifications.
Configuration-
python3 -m opennmt.train \
-gpuid 1 \
-data data-bin_opennmt/en-de \
-save_model cnn-model
The train log-
Loading train dataset from data-bin_opennmt/en-de.train.1.pt, number of examples: 146822

• vocabulary size. source = 6542; target = 8681
  Building model...
  Intializing model parameters.
  NMTModel(
  (encoder): RNNEncoder(
  (embeddings): Embeddings(
  (make_embedding): Sequential(
  (emb_luts): Elementwise(
  (0): Embedding(6542, 500, padding_idx=1)
  )
  )
  )
  (rnn): LSTM(500, 500, num_layers=2, dropout=0.3)
  )
  (decoder): InputFeedRNNDecoder(
  (embeddings): Embeddings(
  (make_embedding): Sequential(
  (emb_luts): Elementwise(
  (0): Embedding(8681, 500, padding_idx=1)
  )
  )
  )
  (dropout): Dropout(p=0.3)
  (rnn): StackedLSTM(
  (dropout): Dropout(p=0.3)
  (layers): ModuleList(
  (0): LSTMCell(1000, 500)
  (1): LSTMCell(500, 500)
  )
  )
  (attn): GlobalAttention(
  (linear_in): Linear(in_features=500, out_features=500, bias=False)
  (linear_out): Linear(in_features=1000, out_features=500, bias=False)
  (sm): Softmax()
  (tanh): Tanh()
  )
  )
  (generator): Sequential(
  (0): Linear(in_features=500, out_features=8681, bias=True)
  (1): LogSoftmax()
  )
  )
  
  
  
  • number of parameters: 21726681
    
    encoder: 7279000
    
    decoder: 14447681
    
    Making optimizer for training.
    
    Stage 1: Keys after executing optim.set_parameters(model.parameters())
    
    optim.optimizer.state_dict()['state'] keys:
    
    optim.optimizer.state_dict()['param_groups'] elements:
    
    optim.optimizer.state_dict()['param_groups'] element: {'params': [139701976939112, 139701976939040, 139701976939256, 139701976939328, 139701976939400, 139701976939472, 139701976939544, 139701976939616, 139701976939688, 139701976939760, 139701976939184, 139701976939904, 139701976939976, 139701976940048, 139701976940120, 139701976940192, 139701976940264, 139701976940336, 139701976940408, 139701976940480, 139702109876296, 139702109876368], 'weight_decay': 0, 'dampening': 0, 'momentum': 0, 'nesterov': False, 'lr': 1.0}
  
  

Start training...

• number of epochs: 13, starting from Epoch 1
• batch size: 64

Loading train dataset from data-bin_opennmt/en-de.train.1.pt, number of examples: 146822
/usr/local/lib/python3.5/dist-packages/OpenNMT_py-0.1-py3.5.egg/onmt/modules/GlobalAttention.py:176: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
align_vectors = self.sm(align.view(batch*targetL, sourceL))
/usr/local/lib/python3.5/dist-packages/torch/nn/modules/container.py:91: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument.
input = module(input)
/usr/local/lib/python3.5/dist-packages/OpenNMT_py-0.1-py3.5.egg/onmt/Loss.py:144: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
return onmt.Statistics(loss[0], non_padding.sum(), num_correct)
Epoch 1, 50/ 2295; acc: 248.00; ppl: 26881171418161356094253400435962903554686976.00; xent: 247255.67; 13724 src tok/s; 0 tgt tok/s; 6 s elapsed
Epoch 1, 100/ 2295; acc: 0.00; ppl: 26881171418161356094253400435962903554686976.00; xent: 4246.80; 14832 src tok/s; 34 tgt tok/s; 10 s elapsed
Epoch 1, 150/ 2295; acc: 100.00; ppl: 26881171418161356094253400435962903554686976.00; xent: 2166.08; 15016 src tok/s; 55 tgt tok/s; 15 s elapsed
Epoch 1, 200/ 2295; acc: 100.00; ppl: 26881171418161356094253400435962903554686976.00; xent: 4949.83; 13395 src tok/s; 20 tgt tok/s; 20 s elapsed
Epoch 1, 250/ 2295; acc: 44.00; ppl: 26881171418161356094253400435962903554686976.00; xent: 11477.15; 13284 src tok/s; 8 tgt tok/s; 25 s elapsed
Epoch 1, 300/ 2295; acc: 0.00; ppl: 26881171418161356094253400435962903554686976.00; xent: 6393.73; 15361 src tok/s; 17 tgt tok/s; 30 s elapsed
Epoch 1, 350/ 2295; acc: 0.00; ppl: 26881171418161356094253400435962903554686976.00; xent: 2211.32; 15890 src tok/s; 49 tgt tok/s; 35 s elapsed
Epoch 1, 400/ 2295; acc: 244.00; ppl: 26881171418161356094253400435962903554686976.00; xent: 11177.75; 12965 src tok/s; 8 tgt tok/s; 40 s elapsed

Also PPL is too high. FYI, I'm using python3 with latest pytorch 0.4.

[da03]

Maybe we need to add dim arguments? In onmt/modules/GlobalAttention.py, change self.sm = nn.Softmax() to self.sm = nn.Softmax(dim=1) and in onmt/ModelConstructor.py change nn.LogSoftmax()) to nn.LogSoftmax(dim=1))? In our latest version these arguments have been added.

[patelrajnath]

I updated the code as you suggested. The issue is still the same as above. I tried the recent code as well but the same issue.

[da03]

oh i think i know the issue now

[da03]

It's not it's not training, it's just that evaluation is wrong. In onmt/Loss.py I changed

         num_correct = pred.eq(target) \
                           .masked_select(non_padding) \
                           .sum()` 

to

num_correct = pred.eq(target) \
                         .masked_select(non_padding) \
                         .long().sum()

and non_padding = target.ne(self.padding_idx) to non_padding = target.ne(self.padding_idx).long()

[patelrajnath]

Now, I'm getting the following error-
Traceback (most recent call last):
File "/HD_4TB_1/Tools/OpenNMT-py/train.py", line 502, in
main()
File "/HD_4TB_1/Tools/OpenNMT-py/train.py", line 494, in main
train_model(model, fields, optim, data_type, model_opt)
File "/HD_4TB_1/Tools/OpenNMT-py/train.py", line 257, in train_model
train_stats = trainer.train(train_iter, epoch, report_func)
File "/HD_4TB_1/Tools/OpenNMT-py/onmt/Trainer.py", line 178, in train
report_stats, normalization)
File "/HD_4TB_1/Tools/OpenNMT-py/onmt/Trainer.py", line 311, in _gradient_accumulation
trunc_size, self.shard_size, normalization)
File "/HD_4TB_1/Tools/OpenNMT-py/onmt/Loss.py", line 123, in sharded_compute_loss
loss, stats = self._compute_loss(batch, **shard)
File "/HD_4TB_1/Tools/OpenNMT-py/onmt/Loss.py", line 207, in _compute_loss
stats = self._stats(loss_data, scores.data, target.view(-1).data)
File "/HD_4TB_1/Tools/OpenNMT-py/onmt/Loss.py", line 142, in _stats
.masked_select(non_padding) \
RuntimeError: Expected object of type torch.cuda.ByteTensor but found type torch.cuda.LongTensor for argument #2 'mask'

[da03]

Oops, let's not change the non_padding at that line, and let's do that right before passing it to onmt.Statistics: return onmt.Statistics(loss.cpu().numpy(), non_padding.long().sum().cpu().numpy(), num_correct.cpu().numpy())

[patelrajnath]

Yes, this solves the issue. Thanks! :)

[da03]

No problem!

[David-Levinthal]

so what was the answer in the end?..How does one get a current checkout to
work with top of tree Pytorch?

On Mon, Apr 16, 2018 at 9:44 AM, Yuntian [e-mail] wrote:

No problem!

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenNMT/OpenNMT-py/issues/403#issuecomment-381670266,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIUuTzzyxWctcURfQwd8f6kEjfbbtywVks5tpMqLgaJpZM4QyorZ
.

[da03]

I'll try to send a PR later. There are two files to modify (@patelrajnath correct me if I was wrong):

1. in onmt/Trainer.py, comment out

if dec_state is not None:
    dec_state.detach()

1. In onmt/Loss.py:

         num_correct = pred.eq(target) \
                           .masked_select(non_padding) \
                           .sum()` 

to

num_correct = pred.eq(target) \
                         .masked_select(non_padding) \
                         .long().sum()

and return onmt.Statistics(loss[0], non_padding.sum(), num_correct) to return onmt.Statistics(loss.cpu().numpy(), non_padding.long().sum().cpu().numpy(), num_correct.cpu().numpy())

Change loss.div(normalization).backward() to loss.div(normalization).backward(retain_graph=True)

[patelrajnath]

I modified the latest checkout as follows-

Step 1- in onmt/Models.py-
Change h.detach_() to h.detach()

Step-2: In onmt/Loss.py,
Change loss.div(normalization).backward() to loss.div(normalization).backward(retain_graph=True)

and
num_correct = pred.eq(target).masked_select(non_padding).sum()
to
num_correct = pred.eq(target).masked_select(non_padding).long().sum()

and
return onmt.Statistics(loss[0], non_padding.sum(), num_correct)
to
return onmt.Statistics(loss[0], non_padding.long().sum(), num_correct)

[David-Levinthal]

thanks..following the discussion got a bit confusing
:-)

On Mon, Apr 16, 2018 at 10:36 AM, Raj Nath Patel [e-mail]
wrote:

I modified the latest checkout as follows-

Step 1- in onmt/Models.py-
Change h.detach_() to h.detach()

Step-2: In onmt/Loss.py,
Change loss.div(normalization).backward() to loss.div(normalization).
backward(retain_graph=True)

and
num_correct = pred.eq(target) \ .masked_select(non_padding) \ .sum()
to
num_correct = pred.eq(target) \ .masked_select(non_padding) \ .long().sum()

and
return onmt.Statistics(loss[0], non_padding.sum(), num_correct)
to
return onmt.Statistics(loss[0], non_padding.long().sum(), num_correct)

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenNMT/OpenNMT-py/issues/403#issuecomment-381687289,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIUuT1Grao4MrSYAuZQf2KT7-wQSyXG-ks5tpNaagaJpZM4QyorZ
.

[patelrajnath]

When I tried using adam optimizer, training ppl is not changing.

[nikhilweee]

@patelrajnath Your changes work fine, except that I got RuntimeError: Expected object of type torch.FloatTensor but found type torch.LongTensor for argument #2 'other' in onmt.Trainer.py line 51
A better solution would be to return numbers instead of tensors. Something like this.

UPDATE: According to https://github.com/pytorch/pytorch/issues/5863#issuecomment-384878245 there's no need to explicitly call .long() on ByteTensors. So this should work just fine.

# onmt/Loss.py line ~139

    def _stats(self, loss, scores, target):
        pred = scores.max(1)[1]
        non_padding = target.ne(self.padding_idx)
        num_correct = pred.eq(target) \
                          .masked_select(non_padding) \
                          .sum().item()
        num_non_padding = non_padding.sum().item()
        return onmt.Statistics(loss.item(), num_non_padding, num_correct)

[David-Levinthal]

Note : out of the box installation of pytorch, invoking the demo preprocessing (works) and then the train command on a gpu still fails
python train.py -data data/demo -save_model demo-model -gpuid 0
bunch of good set output followed by
Start training...

• number of epochs: 13, starting from Epoch 1
• batch size: 64

Loading train dataset from data/demo.train.1.pt, number of examples: 10000
/home/levinth/OpenNMT-py/onmt/Loss.py:144: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
return onmt.Statistics(loss[0], non_padding.sum(), num_correct)
/home/levinth/OpenNMT-py/onmt/Optim.py:125: UserWarning: torch.nn.utils.clip_grad_norm is now deprecated in favor of torch.nn.utils.clip_grad_norm_.
clip_grad_norm(self.params, self.max_grad_norm)
Traceback (most recent call last):
File "train.py", line 502, in
main()
File "train.py", line 494, in main
train_model(model, fields, optim, data_type, model_opt)
File "train.py", line 257, in train_model
train_stats = trainer.train(train_iter, epoch, report_func)
File "/home/levinth/OpenNMT-py/onmt/Trainer.py", line 178, in train
report_stats, normalization)
File "/home/levinth/OpenNMT-py/onmt/Trainer.py", line 321, in _gradient_accumulation
dec_state.detach()
File "/home/levinth/OpenNMT-py/onmt/Models.py", line 611, in detach
h.detach_()
RuntimeError: Can't detach views in-place. Use detach() instead

changing Models.py per PatelRajnath and Loss.py per Nikhilwee results in:
Start training...

• number of epochs: 13, starting from Epoch 1
• batch size: 64

Loading train dataset from data/demo.train.1.pt, number of examples: 10000
Traceback (most recent call last):
File "train.py", line 502, in
main()
File "train.py", line 494, in main
train_model(model, fields, optim, data_type, model_opt)
File "train.py", line 257, in train_model
train_stats = trainer.train(train_iter, epoch, report_func)
File "/home/levinth/OpenNMT-py/onmt/Trainer.py", line 178, in train
report_stats, normalization)
File "/home/levinth/OpenNMT-py/onmt/Trainer.py", line 311, in _gradient_accumulation
trunc_size, self.shard_size, normalization)
File "/home/levinth/OpenNMT-py/onmt/Loss.py", line 124, in sharded_compute_loss
loss.div(normalization).backward()
File "/usr/local/lib/python2.7/dist-packages/torch/tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/__init__.py", line 89, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

[da03]

Have you set loss.div(normalization).backward(retain_graph=True) in onmt/Loss.py?

[David-Levinthal]

Does that cause the back propagation to always go back to the first batch?
I did not do that as it was not part of the "recipe" that had been
discussed .
I will try that and monitor the speed
d

On Tue, May 1, 2018 at 2:36 PM, Yuntian [e-mail] wrote:

Have you set loss.div(normalization).backward(retain_graph=True) in
onmt/Loss.py?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenNMT/OpenNMT-py/issues/403#issuecomment-385798178,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIUuT8UJDDj6mZeV31_Qk2c6jcrEeyorks5tuNVWgaJpZM4QyorZ
.

[David-Levinthal]

That seems to have gotten things going again..and the speed as printed out appears stable through 13 test epochs
Loading train dataset from data/demo.train.1.pt, number of examples: 10000
Epoch 13, 50/ 157; acc: 22.19; ppl: 214.76; xent: 5.37; 18124 src tok/s; 16023 tgt tok/s; 4 s elapsed
Epoch 13, 100/ 157; acc: 21.95; ppl: 204.86; xent: 5.32; 17947 src tok/s; 15403 tgt tok/s; 9 s elapsed
Epoch 13, 150/ 157; acc: 22.89; ppl: 172.13; xent: 5.15; 17386 src tok/s; 14693 tgt tok/s; 14 s elapsed
Train perplexity: 196.24
Train accuracy: 22.2845
Loading valid dataset from data/demo.valid.1.pt, number of examples: 2819
Validation perplexity: 895.563
Validation accuracy: 16.2202
Decaying learning rate to 0.015625

[stefanonardo]

This error occurs to me only on CPU.

[David-Levinthal]

Stefano
Did you do the three sets of patches discussed earlier?

On Mon, May 7, 2018 at 6:07 AM, Stefano Nardo [e-mail]
wrote:

This error occurs to me only on CPU.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/OpenNMT/OpenNMT-py/issues/403#issuecomment-387058434,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIUuT1-vFHjUYJ81WFtEFqtujX8asKirks5twEb8gaJpZM4QyorZ
.

[stefanonardo]

@David-Levinthal I didn't read the whole discussion and I don't want to change my code if it works on GPU. But it's strange that it depends on the chosen device.

[pltrdy]

I'm not sure to understand what going on with those .detach and retain_graph things, but I have some comments / observations.

---

The original error is that we can use detach_() on view, some fixes has been considered:

1. First, replacing h.detach_() with h.detach() is equivalent to doing nothing since h.detach() returns new tensor that we do not keep.
2. Using h = h.detach() would associate the new tensor with the local variable h. It would not change the content of _all (that is immutable) nor any of the objet's parameters. It's useless as well.

Instead of iterating on _all I chose to set the objects parameter 'manually':

    def detach(self):
        self.hidden = tuple([_.detach() for _ in self.hidden])
        self.input_feed = self.input_feed.detach()

This is equivalent to previous code, at least for simple case (not sure if some other cases uses more parameters in _all).

---

We can dodge the previous point (by just removing the detach_() line), or apply suggested fix, in any case we face the following error:

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

For sure, adding retain_graph=True is tempting (since it's suggested), and it seems to solve the problem. But it does not really explain why the whole code was working on 0.3 and is not with 0.4.

Any ideas?

[vince62s]

ok see my comment in #713
Paul your suggestion works fine, even in the case of BPTT.

EDIT:
For the transformer we need to override the detach() function.
I got it working, will include in a PR

[NLPpupil]

I tried all the solutions mentioned above but still failed. Then I give up 0.4 and turn back to 0.3, the program runs fine.

[vince62s]

try to pull from here and test it: https://github.com/Ubiqus/OpenNMT-py
I need to fix label_smoothing.
If something is wrong post the output.

[mmehedin]

Thanks, this works:

try to pull from here and test it: https://github.com/Ubiqus/OpenNMT-py
I need to fix label_smoothing.
If something is wrong post the output.

This issue is closed but it seems there is an additional error that may be useful to put up here:
The following code

I modified the latest checkout as follows-

Step 1- in onmt/Models.py-
Change h.detach_() to h.detach()

Step-2: In onmt/Loss.py,
Change loss.div(normalization).backward() to loss.div(normalization).backward(retain_graph=True)

and
num_correct = pred.eq(target).masked_select(non_padding).sum()
to
num_correct = pred.eq(target).masked_select(non_padding).long().sum()

and
return onmt.Statistics(loss[0], non_padding.sum(), num_correct)
to
return onmt.Statistics(loss[0], non_padding.long().sum(), num_correct)

works up to one point where it gives this error:

/Users/.../Abstractive-summarization-OpenNMT/onmt/Loss.py:98: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
return onmt.Statistics(loss[0], non_padding.long().sum(), num_correct)
/Users/Mihai/Abstractive-summarization-OpenNMT/onmt/Optim.py:69: UserWarning: torch.nn.utils.clip_grad_norm is now deprecated in favor of torch.nn.utils.clip_grad_norm_.
clip_grad_norm(self.params, self.max_grad_norm)
Traceback (most recent call last):
File "train.py", line 320, in
main()
File "train.py", line 316, in main
train_model(model, train, valid, fields, optim)
File "train.py", line 171, in train_model
train_stats = trainer.train(epoch, report_func)
File "/Users/..../Abstractive-summarization-OpenNMT/onmt/Trainer.py", line 149, in train
total_stats.start_time, self.optim.lr, report_stats)
File "train.py", line 94, in report_func
report_stats.output(epoch, batch+1, num_batches, start_time)
File "/Users/..../Abstractive-summarization-OpenNMT/onmt/Trainer.py", line 59, in output
self.ppl(),
File "/Users/..../Abstractive-summarization-OpenNMT/onmt/Trainer.py", line 48, in ppl
return math.exp(min(self.loss / self.n_words, 100))
RuntimeError: Expected object of type torch.FloatTensor but found type torch.LongTensor for argument #2 'other'