Hub: RuntimeError: Module must be applied in the graph it was instantiated for.

[ShellRox]

I'm trying to serve universal sentence encoder with Django.

The code is initialized in the beginning as a background process (by using programs such as supervisor), then it communicates with Django using TCP sockets and eventually returns encoded sentence.

The code:

import socket
from threading import Thread
import tensorflow as tf
import tensorflow_hub as hub
import atexit

# Pre-loading the variables:
embed = hub.Module("https://tfhub.dev/google/universal-sentence-encoder/2")
session = tf.Session()
session.run(tf.global_variables_initializer())
session.run(tf.tables_initializer())
atexit.register(session.close)  # session closes if the script is halted

# TCP socket for communication:
_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
_socket.bind((socket.gethostbyname(socket.gethostname()), 5005))
_socket.listen(1)

# Converts string to vector embedding:
def initiate_connection(conn):
    data = conn.recv(1024)
    conn.send(session.run(embed([data])))
    conn.close()

# Process in background, waiting for TCP message from views.py
while True:
    conn, addr = _socket.accept()
    _thread = Thread(target=initiate_connection, args=(conn,))  # new thread for each request (could be limited later)
    _thread.demon = True
    _thread.start()
    conn.close()

I do this, because loading tables (session.run(tf.tables_initializer())) takes quite a lot of time. Thus if possible, I'd rather load them only once (when server starts).

But I get RuntimeError on conn.send(session.run(embed([data]))):

RuntimeError: Module must be applied in the graph it was instantiated for.

I know something like this has already been asked before, but I'm not sure how could I implement that for pre-initialization purposes.

[vbardiovskyg]

Hi,

it seems like basically the same thing as the issue you are mentioning. I'll copy it over since it is easier to copy than explain :)

# Create graph and finalize (optional but recommended).
g = tf.Graph()
with g.as_default():
  text_input = tf.placeholder(dtype=tf.string, shape=[None])
  embed = hub.Module("https://tfhub.dev/google/universal-sentence-encoder/2")
  my_result = embed(text_input)
  init_op = tf.group([tf.global_variables_initializer(), tf.tables_initializer()])
g.finalize()

# Create session and initialize.
session = tf.Session(graph=g)
session.run(init_op)

# At inference call:
my_result_out = session.run(my_result, feed_dict={text_input: ["Hello world"]})

This also fixes an issue with your code - building a graph at each call of initiate_connection: embed([data]) is actually building a graph.

[ShellRox]

Hi,

it seems like basically the same thing as the issue you are mentioning. I'll copy it over since it is easier to copy than explain :)

# Create graph and finalize (optional but recommended).
g = tf.Graph()
with g.as_default():
  text_input = tf.placeholder(dtype=tf.string, shape=[None])
  embed = hub.Module("https://tfhub.dev/google/universal-sentence-encoder/2")
  my_result = embed(text_input)
  init_op = tf.group([tf.global_variables_initializer(), tf.tables_initializer()])
g.finalize()

# Create session and initialize.
session = tf.Session(graph=g)
session.run(init_op)

```

At inference call:

my_result_out = session.run(my_result, feed_dict={text_input: ["Hello world"]})
`` This also fixes an issue with your code - building a graph at each call ofinitiate_connection:embed([data])` is actually building a graph.

Thank you, it transforms string into embedding vector in milliseconds after initialization.

P.S
Anyone who may use the script above, it won't work. Since sockets pass information using byte data as strings, so you won't really get embedding vector. Therefore it is imperial to serialize your data first, using something powerful (such as Dill).