Addons: Tutorial request for seq2seq greedy and beam decoding, with and withouts attention, and improved documentation

[w4nderlust]

The examples of how to use seq2seq decoders decoders, in particular the beam search one available in the readme https://github.com/tensorflow/addons/tree/master/tensorflow_addons/seq2seq are not sufficient, do not cover combinations of attention and beam, contain undefined variables and in the end are not useful.
There, for instance, decoders __call__ function is used, while in other tutorials, a loop with calls to decoder.step are used, but if you do so with the beam search decoder, the outputs and inputs are not well documented (no description beyond their name), so it is very very hard to understand what is going on, what is the correct way to use them and how to fix errors.

The request then is mainly for a tutorial that shows end to end how to use both greedy and beam search decoders, both with and without attention, that are self contained and work end 2 end and include best practices and abundant comments to explain inputs and outputs.
A secondary request would be to improve the documentation by describing all inputs and outputs parameters, their types, what they are and examples of what they look like.

[guillaumekln]

It seems someone already contributed a tutorial, but I think this could be made simpler and clearer. I can look to propose something different (unless someone else wants to work on this).

[w4nderlust]

It seems someone already contributed a tutorial, but I think this could be made simpler and clearer. I can look to propose something different (unless someone else wants to work on this).

It would be great to add also beam search to it. It doesn't seem to be super straightforward to me, in particular if you want to collect the highest scoring sequence and its logits.
Thank you for the updated docs, very helpful!

[jimthompson5802]

To expand this request

A secondary request would be to improve the documentation by describing all inputs and outputs parameters, their types, what they are and examples of what they look like.

These are some of the apis where more complete documentation would have been helpful, at least for me:

• https://www.tensorflow.org/addons/api_docs/python/tfa/seq2seq/BasicDecoderOutput
• https://www.tensorflow.org/addons/api_docs/python/tfa/seq2seq/BeamSearchDecoderOutput
• https://www.tensorflow.org/addons/api_docs/python/tfa/seq2seq/BeamSearchDecoderState

[guillaumekln]

These are some of the apis where more complete documentation would have been helpful, at least for me

@jimthompson5802 Documentations for these symbols were added in the pull request referenced above (https://github.com/tensorflow/addons/pull/1916).

[abhishek-niranjan]

@w4nderlust I've extended this tutorial to use the same decoder stack and do beam search inference over it using tfa.addons.seq2seq.BeamSearchDecoder. I could create a PR with the updated notebook if you'd like?

[w4nderlust]

@abhishek-niranjan that would certainly be useful, thank you.

[abhishek-niranjan]

@w4nderlust created PR #2140; Added the link to colab notebook in PR description, Hope this works!

[guillaumekln]

The PR has been merged, thanks @abhishek-niranjan! Of course it could always be improved and simplified but I think that the main point has been addressed.