Farm: DistilBert for En, De and Fr

[zolekode]

Question
I see Farm supports german DistilBERT. Is it the case for English and French too ? If yes, how do I use the load those models ?

[tholor]

Sure, you can easily convert models between FARM <-> huggingface's transformers. You can load any of the models from here by just using the model name in a load() method:

LanguageModel.load("distilbert-base-uncased")
Tokenizer.load("distilbert-base-uncased")

In our example scripts, this is as simple as changing this variable to the model name you want:
https://github.com/deepset-ai/FARM/blob/d606df13f7d3d0536de9e5e5de843b2d9422f223/examples/doc_classification.py#L32

This should allow you to use DistilBERT for EN and DE. However, I haven't seen a DistilBERT for french yet.

---

BTW, there's also the option to use an already trained downstream model directly for inference:

inf = Inferencer.load("distilbert-base-uncased-distilled-squad", task_type="question_answering")
# run predictions
QA_input = [{"qas": ["Why is model conversion important?"],
                 "context": "Model conversion lets people easily switch between frameworks."}]
result = inf.inference_from_dicts(dicts=QA_input)

[zolekode]

Thanks works. Would multilingual distilBERT work for French ?

[tholor]

Definitely worth a try! If that doesn't give satisfying results, you could also consider training a french distilBERT yourself (see here)

[zolekode]

Awesome. Thanks