Hub: Universal sentence encoder training

[saikrishna9494]

I want to use the universal sentence encoder for clustering. But the problem is I am having a different domain of data. So Is it possible to train my data to generate the sentence vectors?

[andresusanopinto]

Is your questions whether you can fine-tune the module to your data?

• In general the answer is a fuzzy "yes". Note that not all parts of a module can be fine-tuned (e.g. the vocabulary) and maybe a module does not provides the mechanisms to fine-tune as good as the original (e.g. a way to control dropout or batch norm).

[saikrishna9494]

Can you please send me the snippet of code for fine tuning universal sentence vector with a new document.

[andresusanopinto]

Have a look in: https://www.tensorflow.org/hub/fine_tuning

[saikrishna9494]

Can I create the universal sentence vector model from a different data set other than fine tuning the existing model? Is there any training module available where I can train from my own data from scratch.

[micahprice]

There is a great example showing how to transfer learn the universal sentence encoder module while also training a classifier:
https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/text_classification_with_tf_hub.ipynb

But I am wondering how to just train the module itself to output sentence encodings fine tuned to my dataset, and the fine_tuning page referenced above doesn't make it clear how to do that. My instinct is to train a bogus classifier (real input paragraphs, fake binary classification) and only keep the module, but I would rather do it the correct way.

[vbardiovskyg]

Please take a look at https://github.com/tensorflow/hub/issues/46.

[rsilveira79]

@vbardiovskyg, we are testing the Universal Sentence Encoder model, and wanted to double check where was the large-1 model (https://www.tensorflow.org/hub/modules/google/universal-sentence-encoder-large/1) model trained ? I can relate this model to the models from the original paper

[akhorlin]

It trained same was as the embedding described in the paper and same way as
https://www.tensorflow.org/hub/modules/google/universal-sentence-encoder/1,
only
this version allows for fine-tuning (trainable=True). This embedding was
published in response to #46 https://github.com/tensorflow/hub/issues/46.

On Tue, Jun 5, 2018 at 7:08 PM Roberto Silveira [e-mail]
wrote:

@vbardiovskyg https://github.com/vbardiovskyg, we are testing the
Universal Sentence Encoder model, and wanted to double check where was the
large-1 model (
https://www.tensorflow.org/hub/modules/google/universal-sentence-encoder-large/1)
model trained ? I can relate this model to the models from the original
paper

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/hub/issues/36#issuecomment-394787759, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AbunTA8LfY9eqTBftDqDZDBW8ts9RZmEks5t5rsSgaJpZM4Tc54e
.

[datistiquo]

The question was not answered I think

is the code for the universal sentence encoder actually open-source? I don't want to fine-tune rather than train on my owen crawled data as the opener wanted too?

Is it for german available?

😄