Transformers: What is Synthetic Self-Training?

[hsm207]

The current best performing model on SQuAD 2.0 is BERT + N-Gram Masking + Synthetic Self-Training (ensemble):

What is Synthetic Self-Training?

[thomwolf]

Check Jacob Devlin slides starting from slide 26 here

[hsm207]

@thomwolf thanks, the slides were helpful. Do you know if there is a recording of the talk publicly available somewhere?

[thomwolf]

I don't know!

[CatalinVoss]

It was a very crowded room and these talks are generally not recorded, sorry…

[Dogy06]

Is the synthetic self training module in this code?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jind11]

Same question, will the synthetic self training module be in this code?

[thomwolf]

Presently not in the library and there are no short-term plans to add synthetic self-training to the library.