Transformers: Fine tuned to Multi-choice dataset?

[Qzsl123]

Is it posible to fine tuned to the multi choices problems , which usually has one passage, question and ABCD four options?

[rodgzilla]

Yes it is, the code is not written yet but I'm planning to work on it. The idea is to format the input data the same way the authors of Improving Language Understanding with Unsupervised Learning

You run an inference (context, choice) for each choice, you compute the image of the [CLS] token by a linear layer with 1 output and then compute a softmax over the output of all choices.

I will try to create a PR with this code very soon.

[Qzsl123]

Thx for the reply.
Actually, I have the same plan. But I am not sure whether it will work. Anyway, I will have a try.

[rodgzilla]

If it worked in the OpenAI paper, I don't really see why it wouldn't work with this architecture.

[rodgzilla]

@Qzsl123 The code for multiple choice task is available in PR #96 if you want to test it.

[Qzsl123]

@rodgzilla yeah, I am trying to run it. Thanks for the wonderful job!

[DukeZhu]

Yes it is, the code is not written yet but I'm planning to work on it. The idea is to format the input data the same way the authors of Improving Language Understanding with Unsupervised Learning

You run an inference (context, choice) for each choice, you compute the image of the [CLS] token by a linear layer with 1 output and then compute a softmax over the output of all choices.

I will try to create a PR with this code very soon.

hi，The multi choices problem usually has one passage, question and ABCD four options。In your model, dose context means passage&question ?

[monk1337]

Any update on this issue?