Tensorboard: Embedding visualization with eager execution

[maximilianmozes]

TensorBoard version: 1.10.0
OS: Linux Ubuntu 16.04.5 LTS
Python version: 2.7.12

I am using eager execution and I would like to visualize embeddings in TensorBoard. I use the following code to set up the visualization:

import tensorflow as tf
import tensorflow.contrib.eager as tfe
from tensorflow.contrib.tensorboard.plugins import projector
tf.enable_eager_execution()
tf.executing_eagerly()
self._writer = tf.contrib.summary.create_file_writer('path')
embedding_config = projector.ProjectorConfig()
embedding = embedding_config.embeddings.add()
embedding.tensor_name = self._word_embeddings.name
embedding.metadata_path = 'metadata.tsv'
projector.visualize_embeddings(self._writer, embedding_config)

where self._word_embeddings is my variable for the embeddings. However, when executing this script the following error is thrown:

File "/lib/python2.7/site-packages/tensorflow/contrib/tensorboard/plugins/projector/__init__.py", line 52, in visualize_embeddings
logdir = summary_writer.get_logdir()
AttributeError: 'SummaryWriter' object has no attribute 'get_logdir'

Might this be a bug? Any kind of help is greatly appreciated!

[nfelt]

Hmm, thanks for reporting this. Unfortunately visualize_embeddings() was only designed to work with a tf.summary.FileWriter() for the summary_writer parameter, and so it errors out if you pass in tf.contrib.summary.create_file_writer().

We'll fix this, but for the time being, I would just work around it. I think one workaround would just be to create a tf.summary.FileWriter(path) before calling tf.enable_eager_execution() (since the check that prevents using FileWriter with eager happens on FileWriter construction).

If that doesn't work, a hack to make it work would be just creating a dummy FileWriter object like this:

class DummyFileWriter(object):
  def get_logdir(self):
    return path
projector.visualize_embeddings(DummyFileWriter(), embedding_config)

[maximilianmozes]

Alright, I'll try that then. Thanks for your help and the clarification.

[malikaltakrori]

I tried this. It doesn't work because here the embeddings are not saved. In Graph mode, the embeddings are saved as part of the model (in the checkpoint). Here (the above code, and the workaround) the embeddings are not saved anywhere, just the name, and the metadata.

[nfelt]

@malikaltakrori Yes, you'd still need to write out checkpoints under eager mode yourself, separately. The workaround I suggested was only for fixing the call to projector.visualize_embeddings(), which by design only saves a config file and not the embeddings themselves.

[lanpa]

@maximilianmozes Can you access the embedded vectors as a numpy array? If so, I think tensorboardX might do the trick.

[mmv]

This issue is more relevant now because TF2.0 removes SummaryWriter, making the projector impossible to use without a workaround.