Opennmt-py: Copy Attention Error

[ghost]

When I set -copy_attn:

Traceback (most recent call last):
File "train.py", line 307, in
main()
File "train.py", line 303, in main
trainModel(model, train, valid, fields, optim)
File "train.py", line 166, in trainModel
train_stats = trainEpoch(epoch)
File "train.py", line 141, in trainEpoch
**shard)
File "/home/haixu/Desktop/OpenNMT9.4/onmt/Loss.py", line 175, in computeLoss
unbottle(scores_data), batch, self.tgt_vocab)
File "/home/haixu/Desktop/OpenNMT9.4/onmt/IO.py", line 226, in collapse_copy_scores
scores[:, b, ti] += scores[:, b, offset + i]
IndexError: index 3458 is out of range for dimension 1 (of size 3458)

So, how to set # Genenerator and loss options? What is copy attention layer? What is coverage attention layer? Are there any papers?

[srush]

Sorry, this is a bug. We've been using copy attention in a different branch. I'll fix asap.

[moonlightlane]

I've encountered same issue with the index error. Hope a fix would be published soon. Thanks!!

[srush]

Oh sorry, this should be fixed now in master. Are you still seeing this issue?

[moonlightlane]

@srush So far so good (and looks like copy_attn works better than without it)! Thanks so much for the prompt fix!!

[srush]

sounds good. please let us know if there are more issues. This is an active area of code development, but we are trying to keep it stable and keep up with the research.

[shenfei]

Hi, I encountered a pretty much same problem here when I use -copy_attn in train.py:

/root/OpenNMT-py/onmt/modules/GlobalAttention.py:177: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  align_vectors = self.sm(align.view(batch*targetL, sourceL))
/root/OpenNMT-py/onmt/modules/CopyGenerator.py:91: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  prob = F.softmax(logits)
Traceback (most recent call last):
  File "/root/OpenNMT-py/train.py", line 352, in <module>
    main()
  File "/root/OpenNMT-py/train.py", line 348, in main
    train_model(model, train_dataset, valid_dataset, fields, optim, model_opt)
  File "/root/OpenNMT-py/train.py", line 172, in train_model
    train_stats = trainer.train(epoch, report_func)
  File "/root/OpenNMT-py/onmt/Trainer.py", line 156, in train
    trunc_size, self.shard_size)
  File "/root/OpenNMT-py/onmt/Loss.py", line 120, in sharded_compute_loss
    loss, stats = self._compute_loss(batch, **shard)
  File "/root/OpenNMT-py/onmt/modules/CopyGenerator.py", line 181, in _compute_loss
    batch, self.tgt_vocab)
  File "/root/OpenNMT-py/onmt/io/DatasetBase.py", line 63, in collapse_copy_scores
    scores[:, b, ti] += scores[:, b, offset + i]
IndexError: index 3098 is out of range for dimension 1 (of size 3098)

some additional info:

 * number of train sentences: 1205993
 * number of valid sentences: 4996
 * maximum batch size: 64
 * vocabulary size. source = 2965; target = 3085

And I run preprocess.py with these options defined:

-src_vocab_size 10000 -tgt_vocab_size 10000 -dynamic_dict -max_shard_size 409600

PyTorch version: 0.3.0.post4
OpenNMT-py: latest master branch

I've no clue how to fix this. Did the new changes bring this bug back again or is there something wrong with my settings?
Thanks very much.

[srush]

There is another report of a bug with sharding and copying. I am taking a look.