Flair: Getting tagwise accuracy/fscore metrics in sequence tagger

[mnishant2]

I am training a multi-class sequence tagger. How to get various evaluation ma=etrics like accuracy and fscore for each tag separately?

[CamielK]

https://github.com/zalandoresearch/flair/issues/774#issuecomment-498562991

[alanakbik]

@mnishant2 as @CamielK pointed out you can use the evaluate function of the tagger to do this:

from flair.datasets import ColumnCorpus
from flair.models import SequenceTagger

corpus: ColumnCorpus = ColumnCorpus('path/to/your/corpus', column_format={0: 'text', 1: 'ner'})

tagger: SequenceTagger = SequenceTagger.load('ner')
result, _ = tagger.evaluate(corpus.test)
print(result.detailed_results)

Closing this issue for now, but feel free to reopen if you have more questions!

[Ayush-iitkgp]

A small correction:
Instead of
result, _ = tagger.evaluate(corpus.test)
we need
result, _ = tagger.evaluate([corpus.test])