Tensorboard: WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation
I am unable to build and run TensorBoard from source with bazel. Doing so gives
the warning that the TensorBoard installation is missing (see output in repro
steps).
I'm following the development instructions.
Other issues and the diagnostics output suggest that the issue is with running TensorBoard from the source tree, but that's exactly what I want to do, i.e. make and run local modifications to TensorBoard.
What have I got wrong here?
Thanks!
Environment information (required)
Diagnostics
Diagnostics output
``````
--- check: autoidentify
INFO: diagnose_tensorboard.py version 27fd3ab75df3bb79262c4ef0b9bdf2b993092670
--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='devon-XPS-13-9370', release='4.15.0-70-generic', version='#79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019', machine='x86_64')
INFO: sys.getwindowsversion(): N/A
--- check: package_management
INFO: has conda-meta: False
INFO: $VIRTUAL_ENV: '/home/devon/projects/tensorboard/tf'
--- check: installed_packages
INFO: installed: tb-nightly==2.1.0a20191125
INFO: installed: tf-nightly==2.1.0.dev20191125
INFO: installed: tf-estimator-nightly==2.0.0.dev2019112509
--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.1.0a0'
--- check: tensorflow_python_version
2019-11-25 12:11:03.366248: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2019-11-25 12:11:03.366267: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING: Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING: Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING: Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING: Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING: Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING: Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING: Limited tf.summary API due to missing TensorBoard installation.
INFO: tensorflow.__version__: '2.1.0-dev20191125'
INFO: tensorflow.__git_version__: 'v1.12.1-19144-gf39f4ea'
--- check: tensorboard_binary_path
INFO: which tensorboard: b'/home/devon/projects/tensorboard/tf/bin/tensorboard\n'
--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC =
socket.SOCK_STREAM =
socket.AI_ADDRCONFIG =
socket.AI_PASSIVE =
Loopback flags:
Loopback infos: [(
Wildcard flags:
Wildcard infos: [(
--- check: readable_fqdn
INFO: socket.getfqdn(): 'devon-XPS-13-9370'
--- check: stat_tensorboardinfo
INFO: directory: /tmp/.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=5505471, st_dev=66311, st_nlink=2, st_uid=1000, st_gid=1000, st_size=4096, st_atime=1574194438, st_mtime=1574712658, st_ctime=1574712658)
INFO: mode: 0o40777
--- check: source_trees_without_genfiles
INFO: tensorboard_roots (3): ['', '/home/devon/projects/tensorboard/tensorboard', '/home/devon/projects/tensorboard/tf/lib/python3.6/site-packages']; bad_roots (2): ['', '/home/devon/projects/tensorboard/tensorboard']
--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==0.8.1
astor==0.8.0
cachetools==3.1.1
certifi==2019.9.11
chardet==3.0.4
gast==0.2.2
google-auth==1.7.1
google-auth-oauthlib==0.4.1
google-pasta==0.1.8
grpcio==1.25.0
h5py==2.10.0
idna==2.8
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
Markdown==3.1.1
numpy==1.17.4
oauthlib==3.1.0
opt-einsum==3.1.0
pip==19.3.1
protobuf==3.10.0
pyasn1==0.4.8
pyasn1-modules==0.2.7
requests==2.22.0
requests-oauthlib==1.3.0
rsa==4.0
setuptools==42.0.1
six==1.13.0
tb-nightly==2.1.0a20191125
termcolor==1.1.0
tf-estimator-nightly==2.0.0.dev2019112509
tf-nightly==2.1.0.dev20191125
urllib3==1.25.7
Werkzeug==0.16.0
wheel==0.33.6
wrapt==1.11.2
``````
Suggestion: Avoid tensorboard packages without genfiles
Your Python path contains a tensorboard package that does not
include generated files. This can happen if your current directory
includes the TensorBoard source tree (e.g., you are in the TensorBoard
Git repository). The following directories from your Python path may
be problematic:
- current directory
- '/home/devon/projects/tensorboard/tensorboard' (duplicate underlying directory)
Next steps
Please try each suggestion enumerated above to determine whether it
solves your problem. If none of these suggestions works, please copy
ALL of the above output, including the lines containing only
backticks, into your GitHub issue or comment. Be sure to redact any
sensitive information.
Steps to reproduce (required)
$ mkdir tensorboard
$ cd tensorboard/
$ virtualenv tf
$ source tf/bin/activate
(tf) $ pip install --upgrade pip
(tf) $ pip install tf-nightly
(tf) $ git clone [email protected]:tensorflow/tensorboard.git
(tf) $ cd tensorboard
(tf) $ bazel run //tensorboard -- --logdir=mylogdir
...
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.summary API due to missing TensorBoard installation.
...
dproctor
All 8 comments
Hi @dproctor! Can you clarify: despite the warnings, does TensorBoard
actually work? I usually hit these warnings, too, but TensorBoard still
runs and works fine for me:
$ bazel run //tensorboard -- --logdir=mylogdir
INFO: Analyzed target //tensorboard:tensorboard (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //tensorboard:tensorboard up-to-date:
bazel-bin/tensorboard/tensorboard
INFO: Elapsed time: 0.306s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.compat.v2.summary API due to missing TensorBoard installation.
WARNING:root:Limited tf.summary API due to missing TensorBoard installation.
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
TensorBoard 2.1.0a0 at http://localhost:6006/ (Press CTRL+C to quit)
Ideally, yeah, we wouldn’t have these warnings—just trying to narrow
down whether this is a “spurious warnings are annoying” problem or
“TensorBoard doesn’t actually launch” problem.
If TensorBoard doesn’t actually launch, what does it do? Could you post
any error text or non-zero exit code?
wchargin
on 25 Nov 2019
Thanks for the quick response!
I assumed there was an issue when I saw the warnings and some other github issues referencing these warnings, but, now that I actually poke around with it, it does seem to behave as expected.
So, I think this is really is just a question of annoying spurious warnings.
While it'd be nice to resolve the underlying issue, an easier fix that would have saved me some time would have been if the development instructions were a little more thorough and told the user to expect this.
More generally, I'd love to see other development tips on there, if they exist. For example, is there an easy way to get hot reloading of frontend resources with the bazel config?
dproctor
on 25 Nov 2019
While it'd be nice to resolve the underlying issue,
Looked into this briefly. The warnings are correct: the //tensorboard
binary does not include TF summary support. This is fine, because the
tensorboard process doesn’t actually need to write summaries; it’s
just that TensorFlow is a bit zealous in warning us.
A patch that suffices to silence the warnings is:
diff --git a/tensorboard/BUILD b/tensorboard/BUILD
index a40ff1e8..f4f72841 100644
--- a/tensorboard/BUILD
+++ b/tensorboard/BUILD
@@ -28,6 +28,8 @@ py_binary(
":program",
"//tensorboard:expect_tensorflow_installed",
"//tensorboard/plugins:base_plugin",
+ "//tensorboard/summary",
+ "//tensorboard/summary:tf_summary",
"//tensorboard/uploader:uploader_main_lib",
"//tensorboard/util:tb_logging",
],
This is a bit weird in that the summaries are never actually required,
and it overapproximates the build graph, so if the summary definitions
are changed then bazel run //tensorboard will need to re-compile and
re-link. But that compilation and linking is quite fast (it’s mostly
Python, so it’s basically a no-op), and changes there are also rare. So
maybe this is worth it—thoughts, @nfelt?
an easier fix that would have saved me some time would have been if
the [development instructions] were a little more thorough and told
the user to expect this.[development instructions]: https://github.com/tensorflow/tensorboard/blob/master/DEVELOPMENT.md
Absolutely; I’ll send a PR out presently. Thanks for this feedback.
Please feel free to ask for more clarification or changes as you run
into them.
wchargin
on 26 Nov 2019
For example, is there an easy way to get hot reloading of frontend
resources with the bazel config?
If you’re editing one of TensorBoard’s core plugins, like the scalars
dashboard: not really, I’m afraid. You’ll need to restart the server to
see your changes, because we need to recompile/bundle the HTML. (This is
the step called “Vulcanization” that you may notice in your build logs.)
A bit of a hack to speed this up is to disable JS compilation by
patching tensorboard/components/BUILD:
diff --git a/tensorboard/components/BUILD b/tensorboard/components/BUILD
index 43b1634b..ccf6e5b4 100644
--- a/tensorboard/components/BUILD
+++ b/tensorboard/components/BUILD
@@ -19,7 +19,7 @@ tf_web_library(
tensorboard_html_binary(
name = "index",
- compile = True,
+ compile = False,
input_path = "/tensorboard.html",
output_path = "/index.html",
js_path = "/index.js",
…which will work fine for most plugins, but break some, like the What-If
Tool. You’ll still need to restart the server, but the recompilation
will be much faster.
Now, if you’re working on a new plugin, we encourage you to package and
distribute it via TensorBoard’s dynamic plugin system. With a dynamic
plugin, you can write the frontend with whatever familiar tools you
prefer (e.g., React + Webpack), and you can distribute it as your own
Pip package that TensorBoard will discover and load. Because you can
bring your own stack, hot reloading is compatible here. For more
information, please take a look at our dynamic plugin tutorial,
and let us know if you have any questions.
wchargin
on 26 Nov 2019
Great, thanks for the pointers!
Dynamic plugins are exactly what I was looking for :)
I could probably have found that on my own by spending a little more time digging through the code rather than depending on the documentation, but FWIW, it would have been more discoverable with an explicit mention of the plugin framework in the main README.md or DEVELOPMENT.md.
dproctor
on 26 Nov 2019
this problem appears when i install tensorflow or keras in jupyter .
i solve this problem by create new conda environment and install them from inside new environment
muhammadhassan2
on 14 Dec 2019
For TF 2.1.0, install pip install tensorboard. Solved.
shizukanaskytree
on 4 Apr 2020
It may also due to the version of tensorboard diverses from the tensorflow version.
I solved it by pip uninstall tensorboard and pip install tensorboard==YOUR_TF_VERSION
Erica-Ko
on 31 Dec 2020
Related issues
dniku
·
3Comments
decentralion
·
3Comments
agrinh
·
3Comments
KylePiira
·
4Comments
OverLordGoldDragon
·
3Comments
Most helpful comment
For TF 2.1.0, install
pip install tensorboard. Solved.