Farm: What multi-task strategies are implemented?

[AndriyMulyar]

Question
Suppose I have three tasks A,B,C each with a corresponding head and loss. What multi-task training strategies are implemented?

[tholor]

Hey @AndriyMulyar,

We currently have hard parameter sharing and the simultaneous learning of multiple (auxiliary) tasks (similar to what Sebastian Ruder describes in https://arxiv.org/pdf/1706.05098.pdf). This means in your example the tasks A, B, C would share the parameters of the language model (e.g. BERT) and only the weights from the head are "task-specific". The individual loss of each head is summed up to a total loss and a single backprop is then used to update the weights for each batch. So the model is really learning all tasks at the same time. Changing the aggregation type (e.g. mean) or weighting the loss differently should be straight forward.

We made some good experiences using this type of multi-task learning to make a target task more robust by adding auxiliary tasks. For example:

• You want to classify A. Training on classifying A and B helps to improve on A.
• You want to train a QA model. Training on QA + MaskedLM can help to make QA more robust to domain shifts.

Usage:
From https://github.com/deepset-ai/FARM/releases/tag/0.2.2

1. Add the tasks to the Processor:

    processor = TextClassificationProcessor(...)

    news_categories = ["Sports", "Tech", "Politics", "Business", "Society"]
    publisher = ["cnn", "nytimes","wsj"]

    processor.add_task(name="category", label_list=news_categories, metric="acc", label_column_name="category_label")
    processor.add_task(name="publisher", label_list=publisher, metric="acc", label_column_name="publisher_label")

1. Link the data to right PredictionHead by supplying the task name at initialization:

category_head = MultiLabelTextClassificationHead(layer_dims=[768,5)], task_name="action_type")
publisher_head = MultiLabelTextClassificationHead(layer_dims=[768, 3], task_name="parts")

What type of multi-task strategy are you interested in?

[AndriyMulyar]

Hi, thanks for the response! The obvious approach to me seems to be training each head in a round robin scheme. Are you familiar with any analysis on various strategies for multi-task training? For instance, it is not obvious (to me) how one would perform updates with loss averaging when tasks with different loss functions are present (say cross entropy for a token classification head and MSE for an STS head).

[tholor]

Sure, loss averaging becomes tricky once you have losses with a different scale. You could end up with one dominating task and not learning at all on the others. You could use manual weights or try to automatically infer them (like suggested in this paper and used here).

I have mostly seen MTL with joint losses these days, but I would be very interested to see a comparison to a "round robin" strategy or others. Unfortunately, I am not aware of any paper analyzing this. If you are interested in exploring this, I'll be happy to help you to get this implemented in FARM.

[AndriyMulyar]

I have a custom torch RR dataloader I would be happy to share/integrate. I am working on a related MTL project currently and will see if I can integrate some of these kind of analysis into it. Looking forward to following up.

[tholor]

This would be great!
Happy to discuss / support on the options to integrate it.

[johann-petrak]

How would one go about weighting and combining the per-head losses in a user-specific way?

BTW, I think the issues mentioned here also apply to some degree to multilabel classification where we have the issue of combining per-label losses, different label frequencies etc?

[tholor]

How would one go about weighting and combining the per-head losses in a user-specific way?

@johann-petrak I just drafted an implementation in #220. Would be great to get your feedback on the PR!

BTW, I think the issues mentioned here also apply to some degree to multilabel classification where we have the issue of combining per-label losses, different label frequencies etc.?

Yes, I would say it's related in that sense that in both situations we "aggregate losses" which impacts our loss scale afterwards. However, if you want to adjust multilabel classification loss, I would probably implement it directly in the prediction head: https://github.com/deepset-ai/FARM/blob/master/farm/modeling/prediction_head.py#L434
Do you have any particular adjustments in mind here (other loss function, weighting, scaling ...)?