Hub: Documentation for: using hub.load() for TF1.X modules in TF2

[rjurney]

I realized there is no ticket here but that the work is planned. So I am creating this ticket. At the moment there seems no way to use GloVe/BERT/Elmo embeddings in Tensorflow 2.0 and this is a big bummer.

[vbardiovskyg]

Hi Russel,

we have done considerable work along with the TF team to be able to load old modules via hub.load.

You can load the modules you are mentioing (except GloVe that is not on tfhub.dev), by:

module = hub.load(module_handle, tags=[])
module.signatures["name_of_the_signature_typically_default"](["my sentence"])

But you are right in that this is not documented very well correctly and will be our plan for the following months.

I am changing the name of the issue and leaving open until we write the documentation.

[rjurney]

@vbardiovskyg Thanks, that's great! Sorry, I'm a little slow but what do you do with the module - you iteratively apply it to sentences? Or does it accept an array of sentences?

Previously for Elmo I did this:

# From https://www.depends-on-the-definition.com/named-entity-recognition-with-residual-lstm-and-elmo/
tf.compat.v1.disable_eager_execution()

sess = tf.compat.v1.Session()

elmo_model = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)

sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())

def ElmoEmbedding(x):
    return elmo_model(inputs={
                            "tokens": tf.squeeze(tf.cast(x, tf.string)),
                            "sequence_len": tf.constant(BATCH_SIZE*[MAX_LEN])
                      },
                      signature="tokens",
                      as_dict=True)["elmo"]

text_input = Input(shape=(max_len,), dtype=tf.string)
elmo_embedding = Lambda(ElmoEmbedding, output_shape=(max_len, 1024))(text_input)

What is the role of ["my sentence"] as compared with the previous method of creating a layer?

[rsuwaileh]

@vbardiovskyg Thanks, that's great! Sorry, I'm a little slow but what do you do with the module - you iteratively apply it to sentences? Or does it accept an array of sentences?

Previously for Elmo I did this:

# From https://www.depends-on-the-definition.com/named-entity-recognition-with-residual-lstm-and-elmo/
tf.compat.v1.disable_eager_execution()

sess = tf.compat.v1.Session()

elmo_model = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)

sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())

def ElmoEmbedding(x):
    return elmo_model(inputs={
                            "tokens": tf.squeeze(tf.cast(x, tf.string)),
                            "sequence_len": tf.constant(BATCH_SIZE*[MAX_LEN])
                      },
                      signature="tokens",
                      as_dict=True)["elmo"]

text_input = Input(shape=(max_len,), dtype=tf.string)
elmo_embedding = Lambda(ElmoEmbedding, output_shape=(max_len, 1024))(text_input)

What is the role of ["my sentence"] as compared with the previous method of creating a layer?

I wanted to know the role of ["my sentence"] in the example provided by @vbardiovskyg

[andresusanopinto]

Closing this as documentation on using new TF-2 and legacy hub.Modules has been considerable re-written: https://github.com/tensorflow/hub/blob/master/docs/migration_tf2.md

Additionally in "tensorflow_hub>=0.7", one should be able to use hub.KerasLayer to load old hub Modules. And then use it in eager mode like a function.

In respect to lingering questions:

• In @vbardiovskyg example, ["my sentence"] was an example of a sentence to compute the embedding for. That module accepts a batch of sentences, so ["my sentence 1", "my sentence 2"] would work as well.

The main consideration is: in that example assumes eager mode, in which case one can call the signatures multiple times without penalty. If however one was in graph mode then one should construct a graph that describes the computation and then execute it in a session.

[XMK233]

For legacy TF1 hub.Module, I can easily get the original input and output info with this:

module = hub.Module(handle)
signatures = module.get_signature_names()
for sig in signatures:
    module.get_input_info_dict(sig)
    module.get_output_info_dict(sig)

How can I do the same thing when the TF1 hub.Module is loaded by hub.load() and hub.KerayLayer()? Seems that I don't find API to get the original input and output info when modules are loaded in these ways.

And TF2 SavedModels on TFHub have even empty signature information. Like:

model = hub.load("https://tfhub.dev/google/imagenet/resnet_v2_50/classification/4")
print(list(model.signatures.keys())) ### an empty list [] is printed. 

Does it mean that TF2 SavedModel don't care about the signature any more? Is there still any way for me to get the signature info, original input info and original output info?

Thank you very much.

[arnoegw]

TF2 SavedModels have both signatures (to remain compatible with TF1 SavedModels for serving from C++) and @tf.functions (new in TF2). It's the latter ones that matter: all of TF Hub's tooling for reusing SavedModels in a TensorFlow/Python program (such as hub.KerasLayer) is targeted at the @tf.function __call__ of the restored SavedModel object or its subobjects.

The analogue to multiple signatures of a hub.Module in TF1 are named subobjects of the SavedModel in TF2, which can be accessed like normal Python subobjects: obj.foo, getattr(obj, "foo"). At this time, subobjects are hardly used on TF2 SavedModels on tfhub.dev.

The big new thing in TF2 is the @tf.function object itself. Its exact type is not (to my knowledge) documented by TensorFlow, but certain usage patterns are: see tensorflow.org/guide/function, tensorflow.org/guide/concrete_function and the tf.function API.

In particular, you can check if calling a TF2 SavedModel with certain inputs will succeed by using model.__call__.get_concrete_function(...), where model is the result of hub.load() as in your example, or equivalently, the hub.KerasLayer(...).resolved_object attribute. If not, a ValueError with a pretty good human-readable description of the possible invocations is raised.

Beyond that, be aware that TensorFlow's API stability rules only cover documented symbols; undocumented ones (even if not starting with _) are free to change.

That said, you can help yourself while debugging by looking at things like model.__call__.function_spec.fullargspec for how to call the model.

You can get the various concrete functions ("traces") implementing this tf.function with
model.__call__.get_concrete_function(...) or model.__call__.concrete_functions. Each concrete function cf has attributes cf.structured_input_signature (ignoring those bound to inputs for variables and other resorces) and cf.structured_outputs.