Tensorboard: Embeddings tab: "parsing metadata" hangs

[jkarimi91]

I am trying to visualize embeddings using tensorboard but the embedding tab seems to hang on "parsing metadata".

The code I am using to generate the embeddings and visualization can be found here.

I am running tensorflow 1.2.1 for python 2.7 with gpu support.

[jkarimi91]

I checked the code, metadata tsv file, and the projector_config.ptxt against the tensorboard embedding visualization tutorial ; everything seems to be correct and tensorboard is not giving me any messages in the terminal.

[decentralion]

@dsmilkov Can you please take a look?

[chihuahua]

The metadata_path property within projector_config.pbtxt must be relative to the log directory (logdir). The logdir in this case is named processed. Because projector_config.pbtxt looks like this:

embeddings {
  tensor_name: "embedding:0"
  metadata_path: "processed/vocab_1000.tsv"
}

, the projector plugin attempts to load processed/processed/vocab_1000.tsv, which does not exist. If you set metadata_path to "vocab_1000.tsv", you should be able to successfully load the labels, as I did. Screenshot:

TensorBoard (the embedding projector frontend) misleadingly halts at the "Parsing metadata" step because it attempts to parse the metadata string of "/home/usr/agent007/Desktop/tf-stanford-tutorials/examples/processed/processed/vocab_1000.tsv" not found, or is not a file, which is obviously not valid metadata content - it's just an error message retrieved from the server.

All in all,

• You can solve this problem by excluding the logdir from metadata_path (In your code, remove processed/ from embedding.metadata_path = 'processed/vocab_1000.tsv'.).
• The projector frontend needs to gracefully handle the case in which the tsv file containing metadata cannot be found (instead of misleadingly erring during the parsing stage). I'll create a PR soon that adds a nice message to the embedding projector.

[jkarimi91]

Thanks for looking into this @chihuahua. Quick question, how were you able to retrieve the error message from the server? I saw no output in the terminal when running tensorboard and I could not find any kind of dump file in the log directory:

Also, the Tensorboard Embedding Visualization Tutorial should be updated to reflect the need to set the metadata_path property relative to the log directory.

[wchargin]

@jkarimi91 The version of TensorBoard that you're using doesn't have functioning logging, but we fixed this a while ago in #148; @chihuahua is running from GitHub master, where logging works fine.

[chihuahua]

Per @wchargin, 400/500 responses like that should be printed by werkzeug following #148.

With some prior knowledge of the code, I set a breakpoint here and found that data equalled that error message.

Indeed, the Tensorboard Embedding Visualization Tutorial should be updated.

[chihuahua]

If it is fine with everyone, I am marking this issue closed because #259 got submitted. I have also initiated an internal change to the Tensorboard Embedding Visualization Tutorial that removes LOG_DIR from metadata_path.

[ShariqM]

@chihuahua The documentation still includes the LOGDIR: https://www.tensorflow.org/programmers_guide/embedding

[VXU1230]

@chihuahua This Word2Vec tutorial also includes the logdir. Could you please remove it?

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/tutorials/word2vec/word2vec_basic.py