Hub: Some hub symbols are not available because TensorFlow version is less than 1.14

[tvrbanec]

In Debian, Python3, using pip3 install I got:
sudo pip3 install --upgrade tensorflow

Requirement already up-to-date: tensorflow in /usr/local/lib/python3.7/dist-packages (1.13.1)
Requirement already satisfied, skipping upgrade: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (0.33.1)
Requirement already satisfied, skipping upgrade: six>=1.10.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (1.10.0)
Requirement already satisfied, skipping upgrade: tensorboard<1.14.0,>=1.13.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (1.13.1)
Requirement already satisfied, skipping upgrade: absl-py>=0.1.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (0.7.1)
Requirement already satisfied, skipping upgrade: protobuf>=3.6.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (3.7.1)
Requirement already satisfied, skipping upgrade: gast>=0.2.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (0.2.2)
Requirement already satisfied, skipping upgrade: astor>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (0.7.1)
Requirement already satisfied, skipping upgrade: numpy>=1.13.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (1.16.2)
Requirement already satisfied, skipping upgrade: termcolor>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (1.1.0)
Requirement already satisfied, skipping upgrade: tensorflow-estimator<1.14.0rc0,>=1.13.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (1.13.0)
Requirement already satisfied, skipping upgrade: keras-applications>=1.0.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (1.0.7)
Requirement already satisfied, skipping upgrade: grpcio>=1.8.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (1.19.0)
Requirement already satisfied, skipping upgrade: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.7/dist-packages (from tensorflow) (1.0.9)
Requirement already satisfied, skipping upgrade: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard<1.14.0,>=1.13.0->tensorflow) (0.15.2)
Requirement already satisfied, skipping upgrade: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard<1.14.0,>=1.13.0->tensorflow) (3.1)
Requirement already satisfied, skipping upgrade: setuptools in /usr/lib/python3/dist-packages (from protobuf>=3.6.1->tensorflow) (40.8.0)
Requirement already satisfied, skipping upgrade: mock>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow) (2.0.0)
Requirement already satisfied, skipping upgrade: h5py in /usr/local/lib/python3.7/dist-packages (from keras-applications>=1.0.6->tensorflow) (2.9.0)
Requirement already satisfied, skipping upgrade: pbr>=0.11 in /usr/local/lib/python3.7/dist-packages (from mock>=2.0.0->tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow) (5.1.3)

Now, using code from medium.com, I got warnings:

W0409 15:51:29.103832 140123364300608 __init__.py:56] Some hub symbols are not available because TensorFlow version is less than 1.14
W0409 15:51:29.888326 140123364300608 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/control_flow_ops.py:3632: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:

Why is that happening and how can one avoid it? How could I install newer version with pip, not from git?
Thanks!

[Harshini-Gadige]

Could you please take a look at this https://github.com/tensorflow/hub/issues/263.

Also please try installing 2.0 as below and see if you are running into this issue.
pip install tensorflow==2.0.0-alpha

[tvrbanec]

I did pip3 install --upgrade tensorflow==2.0.0-alpha, but now instead of warning, I got an error:

embed = hub.Module(module_url)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_hub/module.py", line 169, in __init__
    tags=self._tags)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_hub/native_module.py", line 340, in _create_impl
    name=name)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_hub/native_module.py", line 391, in __init__
    self._init_state(name)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_hub/native_module.py", line 394, in _init_state
    variable_tensor_map, self._state_map = self._create_state_graph(name)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_hub/native_module.py", line 451, in _create_state_graph
    import_scope=relative_scope_name)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/saver.py", line 1445, in import_meta_graph
    meta_graph_or_file, clear_devices, import_scope, **kwargs)[0]
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/saver.py", line 1453, in _import_meta_graph_with_return_elements
    raise RuntimeError("Exporting/importing meta graphs is not supported when "
RuntimeError: Exporting/importing meta graphs is not supported when eager execution is enabled. No graph exists when eager execution is enabled.

Where module_url = "https://tfhub.dev/google/universal-sentence-encoder-large/3"

[tvrbanec]

Thanks for referencing to https://github.com/tensorflow/hub/issues/263, because I was not aware that not existing logging is somehow connected with tensorflow.

[tvrbanec]

Now, after uninstalling tensorflow==2.0.0-alpha and installing tensorflow stable again, everything is broken:

import tensorflow_hub as hub
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_hub/__init__.py", line 30, in <module>
    from tensorflow_hub.estimator import LatestModuleExporter
  File "/usr/local/lib/python3.7/dist-packages/tensorflow_hub/estimator.py", line 63, in <module>
    class LatestModuleExporter(tf_v1.estimator.Exporter):
AttributeError: module 'tensorflow_hub.tf_v1' has no attribute 'estimator'

I also tried uninstaling tensorflow-hub and installing it again, but error remains!

[tvrbanec]

Since there is no way to go back to tensorflow==1.13 I installed tensorflow==2.0.0-alpha with change in code:

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

Now, the program is working again, but still with some warnings:

WARNING: Logging before flag parsing goes to stderr.
W0410 14:23:28.022233 140323809154880 deprecation.py:323] From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:63: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.

I will try to recode considering changes presented by https://www.tensorflow.org/alpha/guide/upgrade .

[tvrbanec]

It is a #$%&/() mess. v2 is completely incompatible with v1.

[DennisRauscher]

Hey, maybe this guide can help you migrate to version 2: https://www.tensorflow.org/alpha/guide/migration_guide

[tvrbanec]

Thanks, but migration guide isn't helpful. For example, how would you translate:

with tf.Session() as session:
        session.run([tf.global_variables_initializer(), tf.tables_initializer()])
        sentences_embeddings = session.run(embed(texts))

Migration guide:

Replace tf.Session.run calls
Every tf.Session.run call should be replaced by a Python function.
The feed_dict and tf.placeholders become function arguments.
The fetches become the function's return value.

[captain-pool]

Hi @tvrbanec, if you are using tensorflow==2.0.0-alpha You Can use tf.compat.v1 to replace the variable initializers, sessions, and other non-eager concepts that are by default disabled in TF 2.0.

NOTE: The Following Example Ran Perfectly on:
Python Version: 3.5.2
Tensorflow: 2.0.0-alpha
Tensorflow Hub: 0.4.0

You Can do something similar to this:

with tf.Graph().as_default():
  module_url = "https://tfhub.dev/google/universal-sentence-encoder-large/3"
  module = hub.Module(module_url)
  embed = module( . . . )
  with tf.compat.v1.Session() as sess:
    sess.run(tf.compat.v1.global_variables_initializer())
    sess.run(tf.compat.v1.tables_initializer())
    print(sess.run(embed))

if you don't want to use a new graph for the job, you can always disable eager_execution for the full runtime, and use the default graph.

tf.compat.v1.disable_eager_execution()

[tvrbanec]

I am surprised that people here are interested to help. Thank you!
My problem is that from the list of strings (sentences) that has already been processed, I need to get ELMo embedding of/for sentences.
At the moment i have the following code that works but there are two problems:

1. On the large input corpus of documents (cca 15700 short texts), the execution stops due to OOM (Out-Of-Memory) Error.
2. When i have already installed tersorflow2, it would be nice for my code to follow a new version, and now it is not.

def elmo_base(text):
    import tensorflow.compat.v1 as tf
    tf.disable_v2_behavior()
    import tensorflow_hub as hub
    from sklearn.metrics.pairwise import cosine_similarity

    elmo_url="https://tfhub.dev/google/elmo/2"
    embed = hub.Module(elmo_url, trainable=True)

    embeddings = embed(text)

    with tf.Session() as session:
        session.run([tf.global_variables_initializer(), tf.tables_initializer()])
        sentences_embeddings = session.run(embeddings)
    print('shape:', sentences_embeddings.shape)

    similarity_matrix = cosine_similarity(sentences_embeddings)

    return sentences_embeddings, similarity_matrix

[tvrbanec]

Can I break corpus of documents into two or more subsets and merge embedings? Would that spoil the results?

[captain-pool]

@tvrbanec You can't say anything about spoiling the results unless you try. Since You are running out of memory, that looks like the only way available with your current code and current configuration. Apart from that, I'ld suggest you to look for some cloud alternative. Do also try to create a Dataset Pipeline using tf.data.
And about using Tensorflow 2.0 compliant code, TF Hub provides Pretrained Frozen Computational Graphs. However in TF 2.0 Alpha, Eager Execution is Set By Default. So, Loading up a Graph raises Errors. You need to disable eager execution from the compatibility module of TF 2.0. And use The Graph related Functions like Sessions, Initializers etc. which, as of now, are only available in the compatibility module.( tf.compat.v1)
Let's just hope that, they will be added to TF 2.0 Core, in the upcoming versions.

[arnoegw]

Hi @tvrbanec, I sense a combination of several issues here....

1) The log message "some hub symbols are not available" is correct, but not a problem for you: all symbols for using Hub in TF1 are available as they have always been. You did not need to upgrade. (You can probably pip uninstall and then pip install a version that works for you.)

2) The right way to pass 100,000s of examples through a TensorFlow model is to make batches of dozens (or maybe hundreds). Try putting a for loop inside the with tf.Session() block.

3) TensorFlow 2 is still in alpha, and the backward compatibility story for old Hub modules is not entirely nailed down. Please bear with us while we sort it out for the stable TF 2.0 release.

[tvrbanec]

@arnoegw

1. "(You can probably pip uninstall and then pip install a version that works for you.)"
   Unfortunately, after ones upgrade to 2, uninstall it an install the default version doesn't help as it bring errors. Twighlight zone.
2. Thanks a lot! I tried and it is working very well!

[Ashwinimtech]

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\admin\Downloads\class>python retrain.py --output_graph=retrained_graph.
pb --output_labels=retrained_labels.txt --image_dir=C:\Users\admin\Downloads\cl
ass\Images1
WARNING: Logging before flag parsing goes to stderr.
W0506 21:03:59.432349 9960 __init__.py:56] Some hub symbols are not available b
ecause TensorFlow version is less than 1.14
INFO:tensorflow:Looking for images in 'Angry'
I0506 21:04:00.883151 9960 retrain.py:181] Looking for images in 'Angry'
INFO:tensorflow:Looking for images in 'Calm'
I0506 21:04:01.241952 9960 retrain.py:181] Looking for images in 'Calm'
INFO:tensorflow:Looking for images in 'Happy'
I0506 21:04:01.507152 9960 retrain.py:181] Looking for images in 'Happy'
INFO:tensorflow:Looking for images in 'Sad'
I0506 21:04:01.897153 9960 retrain.py:181] Looking for images in 'Sad'
INFO:tensorflow:Looking for images in 'Yawning'
I0506 21:04:02.131154 9960 retrain.py:181] Looking for images in 'Yawning'
2019-05-06 21:04:04.081157: W tensorflow/core/graph/graph_constructor.cc:1272] I
mporting a graph with a lower producer version 26 into an existing graph with pr
oducer version 27. Shape inference will have run different parts of the graph wi
th different producer versions.
WARNING:tensorflow:From C:\Users\admin\Anaconda3\lib\site-packages\tensorflow\py
thon\ops\control_flow_ops.py:3632: colocate_with (from tensorflow.python.framewo
rk.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
W0506 21:04:06.374361 9960 deprecation.py:323] From C:\Users\admin\Anaconda3\li
b\site-packages\tensorflow\python\ops\control_flow_ops.py:3632: colocate_with (f
rom tensorflow.python.framework.ops) is deprecated and will be removed in a futu
re version.
Instructions for updating:
Colocations handled automatically by placer.
INFO:tensorflow:Saver not created because there are no variables in the graph to
restore
I0506 21:04:13.612774 9960 saver.py:1483] Saver not created because there are n
o variables in the graph to restore
WARNING:tensorflow:From C:\Users\admin\Anaconda3\lib\site-packages\tensorflow\py
thon\ops\losses\losses_impl.py:209: to_float (from tensorflow.python.ops.math_op
s) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
W0506 21:04:14.657976 9960 deprecation.py:323] From C:\Users\admin\Anaconda3\li
b\site-packages\tensorflow\python\ops\losses\losses_impl.py:209: to_float (from
tensorflow.python.ops.math_ops) is deprecated and will be removed in a future ve
rsion.
Instructions for updating:
Use tf.cast instead.
2019-05-06 21:04:14.751576: I tensorflow/core/platform/cpu_feature_guard.cc:141]
Your CPU supports instructions that this TensorFlow binary was not compiled to
use: AVX2
INFO:tensorflow:100 bottleneck files created.
I0506 21:04:32.224606 9960 retrain.py:474] 100 bottleneck files created.
INFO:tensorflow:200 bottleneck files created.
I0506 21:04:33.691009 9960 retrain.py:474] 200 bottleneck files created.
INFO:tensorflow:300 bottleneck files created.
I0506 21:04:34.299410 9960 retrain.py:474] 300 bottleneck files created.
INFO:tensorflow:400 bottleneck files created.
I0506 21:04:34.736211 9960 retrain.py:474] 400 bottleneck files created.
INFO:tensorflow:500 bottleneck files created.
I0506 21:04:35.391412 9960 retrain.py:474] 500 bottleneck files created.
INFO:tensorflow:600 bottleneck files created.
I0506 21:04:36.483414 9960 retrain.py:474] 600 bottleneck files created.
INFO:tensorflow:Creating bottleneck at /tmp/bottleneck\Calm\resized_man-with-bri
stle-on-calm-face-nature-background-defocused-man-with-beard-or-unshaven-guy-loo
ks-handsome-outdoor-guy-bearded-and-attractive-cares-about-his-appea_0039.jpg_ht
tps~tfhub.dev~google~imagenet~inception_v3~feature_vector~1.txt
I0506 21:04:36.655014 9960 retrain.py:354] Creating bottleneck at /tmp/bottlene
ck\Calm\resized_man-with-bristle-on-calm-face-nature-background-defocused-man-wi
th-beard-or-unshaven-guy-looks-handsome-outdoor-guy-bearded-and-attractive-cares
-about-his-appea_0039.jpg_https~tfhub.dev~google~imagenet~inception_v3~feature_v
ector~1.txt
WARNING:tensorflow:From retrain.py:359: FastGFile.__init__ (from tensorflow.pyth
on.platform.gfile) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.gfile.GFile.
W0506 21:04:36.655014 9960 deprecation.py:323] From retrain.py:359: FastGFile._
_init__ (from tensorflow.python.platform.gfile) is deprecated and will be remove
d in a future version.
Instructions for updating:
Use tf.gfile.GFile.
Traceback (most recent call last):
File "retrain.py", line 1333, in
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "C:\Users\admin\Anaconda3\lib\site-packages\tensorflow\python\platform\ap
p.py", line 125, in run
_sys.exit(main(argv))
File "retrain.py", line 1049, in main
bottleneck_tensor, FLAGS.tfhub_module)
File "retrain.py", line 469, in cache_bottlenecks
resized_input_tensor, bottleneck_tensor, module_name)
File "retrain.py", line 411, in get_or_create_bottleneck
bottleneck_tensor)
File "retrain.py", line 368, in create_bottleneck_file
with open(bottleneck_path, 'w') as bottleneck_file:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/bottleneck\Calm\
resized_man-with-bristle-on-calm-face-nature-background-defocused-man-with-beard
-or-unshaven-guy-looks-handsome-outdoor-guy-bearded-and-attractive-cares-about-h
is-appea_0039.jpg_https~tfhub.dev~google~imagenet~inception_v3~feature_vector~1.
txt'

same error please help me.

[samiratzn]

I can not import hub module .
windows 10
Tensorflow version 2.0.0-alpha0
Python 3

---

ModuleNotFoundError Traceback (most recent call last)
~\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_hub\tf_v1.py in
28 try:
---> 29 from tensorflow.compat.v1 import * # pylint: disable=wildcard-import
30 # The previous line also gets us tensorflow.compat.v1.estimator.

ModuleNotFoundError: No module named 'tensorflow.compat.v1'

During handling of the above exception, another exception occurred:

ImportError Traceback (most recent call last)
in
----> 1 import tensorflow_hub as hub

~\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_hub__init__.py in
28 # error message is thrown instead of an obscure error of missing
29 # symbols at executing the imports.
---> 30 from tensorflow_hub.estimator import LatestModuleExporter
31 from tensorflow_hub.estimator import register_module_for_export
32 from tensorflow_hub.feature_column import image_embedding_column

~\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_hub\estimator.py in
23 from absl import logging
24 import tensorflow as tf
---> 25 from tensorflow_hub import tf_utils
26 from tensorflow_hub import tf_v1
27

~\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_hub\tf_utils.py in
26 import tensorflow as tf
27
---> 28 from tensorflow_hub import tf_v1
29
30

~\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow_hub\tf_v1.py in
31 # Be sure not to import from tensorflow_estimator without version selection.
32 except ImportError:
---> 33 from tensorflow import add_to_collection
34 from tensorflow import app
35 from tensorflow import assign

ImportError: cannot import name 'add_to_collection'

[AnnettAnd]

Hi,
Did you install "tf-nightly"? That helped me.
BR