Transformers: Fine-tuning T5 in Tensorflow

[tqdo]

I wonder whether there is an example/tutorial about how to fine-tune T5 for a particular task (say translation) in Tensorflow. On the doc website it says that This model is a tf.keras.Model tf.keras.Model sub-class. However I am not really sure what the output is for the Keras model. It seems like the output (the target sequence, in the case of translation) are considered input for the T5 model as well.

Thank you in advance.

[patrickvonplaten]

Hi @tqdo,

Good question. We will add better explanation for TF T5 soon :-)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[wangcongcong123]

@tqdo. I open sourced fine-tuning T5 by customizing training loop in Tensorflow2.0+: https://github.com/wangcongcong123/ttt
Hope this helps.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.