Fairseq: Any performance comparison between pre-norm and post-norm for Transformer on Machine Translation

Created on 17 Jun 2020  路  3Comments  路  Source: pytorch/fairseq

As the name implies, can you provide any performance comparison between pre-norm and post-norm performance comparison using a transformer on Machine Translation Dataset?

question
Source
picture gaopengcuhk

Most helpful comment

As the name implies, can you provide any performance comparison between pre-norm and post-norm performance comparison using a transformer on Machine Translation Dataset?

You can refer to the ACL2019 paper https://arxiv.org/abs/1906.01787, which is based on fairseq.

picture SunbowLiu on 22 Jun 2020
👍2

All 3 comments

What is the pre-norm and post-norm?

Bachstelze picture Bachstelze on 21 Jun 2020

As the name implies, can you provide any performance comparison between pre-norm and post-norm performance comparison using a transformer on Machine Translation Dataset?

You can refer to the ACL2019 paper https://arxiv.org/abs/1906.01787, which is based on fairseq.

SunbowLiu picture SunbowLiu on 22 Jun 2020
👍2

Have a look at https://github.com/wangqiangneu/dlcl to reproduce the results.

Bachstelze picture Bachstelze on 25 Jun 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

mjpost picture mjpost  路  3Comments
neel04 picture neel04  路  3Comments
ashim95 picture ashim95  路  3Comments
zhaoxv picture zhaoxv  路  3Comments
kyquang97 picture kyquang97  路  3Comments