Hub: Semantic similarity gets slower in a for loop

[dav-ell]

I have a function like this

def sentences_to_vectors(sentences, embed=None):
    if embed == None:
        embed = hub.Module("https://tfhub.dev/google/universal-sentence-encoder-large/3")

    with tf.Session() as session:
        session.run([tf.global_variables_initializer(), tf.tables_initializer()])
        message_embeddings = session.run(embed(sentences))

        return message_embeddings

And I run it like this:

import tensorflow as tf
import tensorflow_hub as hub

embed = hub.Module("https://tfhub.dev/google/universal-sentence-encoder-large/3")
# arr_of_arr_of_strings = [ [ "hello world", "hi" ], [ "foo bar", "giraffe" ], ... ]
for arr_of_strings in arr_of_arr_of_strings:
    vectors = sentences_to_vectors(arr_of_strings, embed=embed)

This works great for the first few iterations (about 12-13 seconds with len(arr_of_strings) == 2500). However, by iteration 50, the time to run sentences_to_vectors is over one minute.

I assume I'm doing something wrong here (some state I need to reset between calls), but it's not immediately clear to me. Anyone know how to fix this, or if this has to do with the TF-Hub module?

[dav-ell]

I was able to fix this by changing the function to

def sentences_to_vectors(sentences):
    embed = hub.Module("https://tfhub.dev/google/universal-sentence-encoder-large/3")
    with tf.Session() as session:
        session.run([tf.global_variables_initializer(), tf.tables_initializer()])
        message_embeddings = session.run(embed(sentences))

    tf.reset_default_graph()
    return message_embeddings

It appears that running the embedder module across multiple sessions resulted in a steadily increasing graph size for TF to compute. The fix simply loads the module and clears it every iteration. There may be a better way to do this that doesn't require re-loading the module every time (which may also reduce some of the computational advantages TF can use). I'm leaving this issue open for now in case someone has a better solution.

[Harshini-Gadige]

Thanks for sharing the work-around. We will leave this open for better solutions.

[vbardiovskyg]

Hi,

this is still building the graph every time a request comes. Please take a look at: https://github.com/tensorflow/hub/blob/master/docs/common_issues.md#running-inference-on-a-pre-initialized-module

[MyBruso]

Hello, did anyone get this embedding working efficiently in TF2.0? I am trying to get embedding of input texts in a loop and it is still very slow, taking ~ 5 sec for each input.