Farm: Predicting without persisting / using different labels for different prediction heads (multitask)

[Garstig]

Hi everyone!

I just started to work with FARM and have two questions about it.

1. How do I use a model without storing it first? In all your examples you first train a model, save it to disk and then load it via an inferencer. But what do I do, if I want to just use my text classification model right after training it? Right now the inferencer does not support multiple prediction heads, so there must be a way to do so I guess? I just can't find the correct method for this.

2. How can I add multiple prediction heads for different text classification tasks? As far as I see it, many blocks in the process are referring to the data silo, like the trainer, the optimizer and also the prediction_head. But every data silo only has one processor, and every processor needs one label column. But for different text classification tasks, I want to have different label columns. Is this possible at all?

I like the idea behind your framework a lot! It would be cool if someone could help me to understand my problems better :). Right now I have a workaround for my problems but it consists of ugly code and has a big performance issue.

Best regards,
Garstig

[tholor]

Hi @Garstig,

Happy to answer your questions:

How do I use a model without storing it first? In all your examples you first train a model, save it to disk and then load it via an inferencer. But what do I do, if I want to just use my text classification model right after training it?

Instead of calling Inferencer.load(..) you can just use the regular constructor:

inferencer = Inferencer(model=model, processor=processor,batch_size=4, gpu=False)
basic_texts = [
    {"text": "Some text you want to classify"},
    {"text": "A second sample"}
]
preds = inferencer.inference_from_dicts(dicts=basic_texts)

How can I add multiple prediction heads for different text classification tasks?

We have the concept of "tasks" for this purpose. By default, one processor only generates the labels for one task (using label_list and label_column_name). As this is the most common use case, we decided to "hide" the complexity of tasks within the processor:

https://github.com/deepset-ai/FARM/blob/fb6370ad5d99efa0b9126d72ce87c872234fff45/farm/data_handler/processor.py#L376-L388

However, you can easily adjust a processor to generate labels for multiple tasks. Just initialize it without the args label_list and metric and add tasks afterwards:

processor = TextClassificationProcessor(tokenizer=tokenizer, 
                                            max_seq_len=128,
                                            data_dir="samples/doc_class",
                                            train_filename="train-sample.tsv")    

processor.add_task(name="task_a", label_list=["some_label", "another"], metric="acc", label_column_name="column_a", task_type="classification")

processor.add_task(name="task_b", label_list=["more", "labels"], metric="acc", label_column_name="column_b", task_type="classification")

Afterwards you can create multiple prediction heads and link them to your processor by supplying the task_name:

head_a = TextClassificationHead(layer_dims=[768, 2], task_name="task_a")
head_b = TextClassificationHead(layer_dims=[768, 2], task_name="task_b")

model = AdaptiveModel(
    language_model=language_model,
    prediction_heads=[head_a, head_b],
    lm_output_types=["per_sequence", "per_sequence"],
    ...
)

Although this approach is quite powerful, we know that this workflow is probably not the most intuitive and we are still considering redesigning this part of the framework.

Right now the inferencer does not support multiple prediction heads, so there must be a way to do so I guess?

We haven't implemented multihead support in the inferencer yet. However, it should be relatively straight forward to change this. It's mainly due to the prediction_type attribute that is not correctly set in case of multiple heads.

https://github.com/deepset-ai/FARM/blob/fb6370ad5d99efa0b9126d72ce87c872234fff45/farm/infer.py#L77-L84

If that's something needed for your use case, we would be happy to see either a PR from you or we implement it (might take a couple of days though).

Hope this helps?!

[Garstig]

Hi @tholor!

Thanks for your response! I think it perfectly answers my questions.

I now realized what an inferencer is in context of deep learning. Before I totally misunderstood the use case of the class because I didn't know the name.

If I will actually use multiple prediction heads I will make a PR.

Thanks again and have a nice day!