Farm: Could I extract embeddings from the model which has been fine tuned ?

[JiangYanting]

Question
Could I extract embeddings from the model which has been fine tuned ?

Additional context
Hello, I have fine tuned the original "pytorch_model.bin" using the script "doc_classification_multilabel.py". And it produced a new language model "language_model.bin" and many other files. So I'm wondering whether I can extract embeddings of sentences from this newly-formed language model "language_model.bin"? Thank you !

[tholor]

Hi @JiangYanting ,

Sure, please check out the example in examples/embedding_extraction.py. I just updated it to reflect the latest changes in the Inferencer.

You can now load a) fine-tuned models from a directory and b) any model in transformer format from their model hub .

Hope this helps.