Hub: Use hub.text_embedding_column together with tf.keras.layers.DenseFeatures

[dfalbel]

Here's a small reproducible code with TF 2.0:

import tensorflow as tf
import tensorflow_hub as hub
feature_columns = [hub.text_embedding_column(key = "text", module_spec = "https://tfhub.dev/google/elmo/2")]
embedding = tf.keras.layers.DenseFeatures(feature_columns = feature_columns)

Errors with:

ValueError: Items of feature_columns must be a FeatureColumn. Given (type <class 'tensorflow_hub.feature_column._TextEmbeddingColumn'>): _TextEmbeddingColumn(key='text', module_spec=<tensorflow_hub.native_module._ModuleSpec object at 0x7f8efcd58f60>, trainable=False).

Here's the same code in colab:

https://colab.research.google.com/drive/1MDFr9fhUYO3QSDayVrBjJBehMzZH8Otq

[vbardiovskyg]

Hi Daniel,

making hub feature_column compatible with the FeatureColumnV2 (FeatureColumn as opposed to _FeatureColumn) is a relatively new feature, a crucial part is implemented here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/feature_column/feature_column_v2.py#L312.

The reason why it fails for you is simply that the feature didn't get into TF2.0 beta release. Trying with !pip install tf-nightly-2.0-preview should work.

[dfalbel]

Thanks very much @vbardiovskyg !
I wonder if it's just the daily buiuld that is not working, but I get a different error when I try to use the created layer:

RuntimeError: variable_scope dense_features_9/text_hub_module_embedding/module/ was unused but the corresponding name_scope was already taken.

Again, this is the complete code:

import tensorflow as tf
import tensorflow_hub as hub
feature_columns = [hub.text_embedding_column(key = "text", module_spec = "https://tfhub.dev/google/nnlm-en-dim128/1")]
embedding = tf.keras.layers.DenseFeatures(feature_columns = feature_columns)
input = {'text': tf.keras.layers.Input(shape = [1,], dtype = 'string')}
output = embedding(input)

And the colab link: https://colab.research.google.com/drive/1MDFr9fhUYO3QSDayVrBjJBehMzZH8Otq
I tried with other models in thub.dev too and got the same error.

[rjurney]

@dfalbel Were you able to get this working? Did you see #281?

[dfalbel]

@rjurney Not yet. I keep having these RuntimeError: variable_scope module_1/ was unused but the corresponding name_scope was already taken.

See this colab notebook: https://colab.research.google.com/drive/1MDFr9fhUYO3QSDayVrBjJBehMzZH8Otq

[vbardiovskyg]

Hi Daniel and Russel, you are right, this actually seems broken now.

[galfridman]

Any news regarding the bug or a workaround?

[rjurney]

Yeah, it would be cool if this worked in 2.0.1