Opennmt-py: Correction needed in opts.py for option -copy_attn_force

[ratishsp]

Hi,
Currently the help statement in opts.py for -copy_attn_force says 'When available, train to copy'. I think it should instead be something along the lines of the relevant comment in source code 'Copy only probability for not-copied tokens'.
Thanks.

[srush]

Okay, we will make this documentation more clear. Are you using this feature? We thought it would be useful, but it might be better to just remove.

[ratishsp]

Okay. I am not using the feature yet. I was unsure about what it is and then tried to study the code. You may remove it if it is not needed.

[parajain]

I couldn't understand the use of -copy_attn_force, also -reuse_copy_attn means we will use attention layer which is used by the decoder?
Can you give some basic explanation or please point to relevant papers?

Thanks for the contribution, it is really helpful.

Thanks

[ratishsp]

Yes, -reuse_copy_attn means we reuse the attention distribution as the copy distribution. It is the approach followed by See et al in "Get to the point: Summarization with pointer-generator networks."

[parajain]

Thanks @ratishsp, and what about -copy_attn_force ? I am not aware of any reference which trains a separate copy layer, is that a better idea?

[ratishsp]

From my reading of the code in CopyGenerator.py, I see that in -copy_attn_force, you have copy probabilities of source tokens and gen probabilities of non-copied tokens. Without -copy_attn_force, you have copy probabilities of source tokens and gen probabilities of non-unk target tokens. As to your second question, Gu, Jiatao, et al. "Incorporating copying mechanism in sequence-to-sequence learning." uses separate copy layer. If that is a better idea has to be empirically verified.

[vince62s]

@ratishsp if you get a chance can you PR an addendum to the doc sstring ? thanks.