Flair: early stop according to dev set performance

[ciaochiaociao]

Hello,

Thank you for your fantastic work of creating flair model. I have got a question here.

Is it possible to early stop according to the dev set performance (say, f1 score of WNUT-17 dev set) instead of the training loss?

Best Regards

[lucaventurini]

Hi,

I think it should be already like that. Try to have look at #1250 and related.

[alanakbik]

Hi @ciaochiaociao, @lucaventurini is right, if you set train_with_dev=False in the ModelTrainer (which is the default), annealing is done against dev F1 score. Only if you set train_with_dev=True you anneal using training loss.

[ciaochiaociao]

Thank you all. I did use an example that turns on train_with_dev. After turning it off, it works as what I want.