Parlai: Do you consider adding transformer to agents?

[zhaoguangxiang]

Transformer works well on NMT. I wonder why it is not used in dialogue generation .

[stephenroller]

We are using it, but have not made our implementation public yet. We expect to have a public version you can "beta test" in the 1-2 week range.

[stephenroller]

In the meantime, you can use fairseq's implementation to play with using -m fairseq --arch transformer

[stephenroller]

Hmm, looks like there's a problem with an empty batch somewhere. Can you give the full command you ran?

[ricsinaruto]

Btw, I used Transformer for dialog modelling, check my Seq2seqChatbots repository.
However, I used the official tensorflow implementation, it would be cool if it was integrated into ParlAI.

I just found the pull request related to the Transformer, looks like it's very close to being merged!

[stephenroller]

Yeah should be merging in #1283 later today.