Flair: Load a model and run Inference only

[alioz1967]

Hi, I would like to do inference only with the code in
https://github.com/zalandoresearch/flair/blob/d6a761a88668eeb695b0b0e53b1a15ddaf9c2524/resources/docs/TUTORIAL_7_TRAINING_A_MODEL.md#training-a-sequence-labeling-model

To skip the training, I tried to load my previous model 'resources/taggers/example-ner/final-model.pt' with torch.load but then final_test function doesn't work.

What is the best way to only run inference on the test set with a previously run model, please? Thank you & Regards. Ali.

[Rexhaif]

Hi, i suggest you to try following approach:

from flair.models import SequenceTagger
model = SequenceTagger.load_from_file("resources/taggers/example-ner/final-model.pt")

[alanakbik]

Yes, though in the newest version, you simply use load(), i.e.

from flair.models import SequenceTagger
model = SequenceTagger.load("resources/taggers/example-ner/final-model.pt")

[alioz1967]

@alanakbik I managed to load the model thanks to your command, Thank you very much for your replies! Would you please see below, not sure why I'm getting this error? When the inference (final_test) is run as part of the trainer.py file, I don't get this issue.
Would you please advise me on how to run only inference if the method below is incorrect? I'm trying to reach the same output (F1 scores and the confusion matrix as your training tutorial # 7). Many thanks in advance.
```
import torch
import os
from flair.data import Corpus
from flair.datasets import ColumnCorpus
from flair.embeddings import TokenEmbeddings, WordEmbeddings, StackedEmbeddings,CharacterEmbeddings, FlairEmbeddings
from typing import List
from flair.models import SequenceTagger

columns = {0: 'text', 1: 'pos', 2: 'ner'}

data_folder = './newfolder'

corpus: Corpus = ColumnCorpus(data_folder, columns,
train_file='train.txt',
test_file='test.txt')#,
#dev_file='dev.txt')

2. what tag do we want to predict?

tag_type = 'ner'

tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

embedding_types: List[TokenEmbeddings] = [

FlairEmbeddings('multi-forward'),
FlairEmbeddings('multi-backward'),

]

embeddings: StackedEmbeddings = StackedEmbeddings(embeddings=embedding_types)

model = SequenceTagger.load("resources/taggers/example-ner/final-model.pt")

model.evaluate(corpus.test)```

```2019-10-23 21:16:34,575 Train: newfolder/train.txt
2019-10-23 21:16:34,576 Dev: None
2019-10-23 21:16:34,576 Test: newfolder/testnew.txt

---

AttributeError Traceback (most recent call last)
in
33 model = SequenceTagger.load("resources/taggers/example-ner/final-model.pt")
34
---> 35 model.evaluate(corpus.test)

.../lib/python3.7/site-packages/flair/models/sequence_tagger_model.py in evaluate(self, data_loader, out_path, embeddings_storage_mode)
266
267 with torch.no_grad():
--> 268 features = self.forward(batch)
269 loss = self._calculate_loss(features, batch)
270 tags, _ = self._obtain_labels(features, batch)

.../lib/python3.7/site-packages/flair/models/sequence_tagger_model.py in forward(self, sentences)
405 self.embeddings.embed(sentences)
406
--> 407 lengths: List[int] = [len(sentence.tokens) for sentence in sentences]
408 longest_token_sequence_in_batch: int = max(lengths)
409

.../lib/python3.7/site-packages/flair/models/sequence_tagger_model.py in (.0)
405 self.embeddings.embed(sentences)
406
--> 407 lengths: List[int] = [len(sentence.tokens) for sentence in sentences]
408 longest_token_sequence_in_batch: int = max(lengths)
409

AttributeError: 'Token' object has no attribute 'tokens'```

[alanakbik]

@alioz1967 you need to pass a DataLoader to the evaluate() method. In the DataLoader, you can set parameters like batch_size to influence how many sentences are tagged as once. Try this code:

# import data loader
from flair.datasets import DataLoader

# wrap test data with data loader
test_dataloader = DataLoader(corpus.test, batch_size=32)

# evaluate
tagger.evaluate(test_dataloader)

[alioz1967]

@alanakbik Dear Alan, Thank you for your reply. I ran the commands as below. but still can't see the micro macro F1 scores and the confusion matrix, the things I used to see after the training with final_test

---

test_dataloader = DataLoader(corpus.test, batch_size=128)
model.evaluate((test_dataloader), out_path= "./out.txt" )

(<flair.training_utils.Result at 0x7ffa2658ee80>,
 tensor(287897.8750, device='cuda:0'))

[alanakbik]

Hello @alioz1967 you can print results like this:

result, main_score = tagger.evaluate(test_dataloader)
print(result.detailed_results)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.