Keras: callbacks.TensorBoard can't write images with conv layers.

[mbinkowski]

callbacks.TensorBoard produces an error while used withConv1D or Conv2D layers and write_images enabled. Running the code

import keras, os
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)
batch_size = 32

nn = keras.models.Sequential()
nn.add(keras.layers.core.Reshape((28, 28, 1), input_shape=(784,)))
nn.add(keras.layers.convolutional.Conv2D(filters=4, kernel_size=3))
nn.add(keras.layers.core.Flatten())
nn.add(keras.layers.core.Dense(units=10, activation='softmax'))

nn.compile(optimizer='rmsprop', loss='categorical_crossentropy')

log_dir='./logs'
if not os.path.exists(log_dir):
    os.makedirs(log_dir)
tb = keras.callbacks.TensorBoard(log_dir='./logs', write_images=True, histogram_freq=1)

nn.fit(mnist.train.images, mnist.train.labels,
       callbacks=[tb],
       validation_data=(mnist.test.images, mnist.test.labels),
       batch_size=batch_size)

Yields the following error message:

Using TensorFlow backend.
Extracting /tmp/data/train-images-idx3-ubyte.gz
Extracting /tmp/data/train-labels-idx1-ubyte.gz
Extracting /tmp/data/t10k-images-idx3-ubyte.gz
Extracting /tmp/data/t10k-labels-idx1-ubyte.gz
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "BestSplits" device_type: "CPU"') for unknown op: BestSplits
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "CountExtremelyRandomStats" device_type: "CPU"') for unknown op: CountExtremelyRandomStats
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "FinishedNodes" device_type: "CPU"') for unknown op: FinishedNodes
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "GrowTree" device_type: "CPU"') for unknown op: GrowTree
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "ReinterpretStringToFloat" device_type: "CPU"') for unknown op: ReinterpretStringToFloat
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "SampleInputs" device_type: "CPU"') for unknown op: SampleInputs
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "ScatterAddNdim" device_type: "CPU"') for unknown op: ScatterAddNdim
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TopNInsert" device_type: "CPU"') for unknown op: TopNInsert
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TopNRemove" device_type: "CPU"') for unknown op: TopNRemove
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "TreePredictions" device_type: "CPU"') for unknown op: TreePredictions
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_kernel.cc:943] OpKernel ('op: "UpdateFertileSlots" device_type: "CPU"') for unknown op: UpdateFertileSlots
Train on 55000 samples, validate on 10000 samples
Epoch 1/10
54784/55000 [============================>.] - ETA: 0s - loss: 0.3635Traceback (most recent call last):
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\client\session.py", line 1022, in _do_call
    return fn(*args)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\client\session.py", line 1004, in _run_fn
    status, run_metadata)
  File "C:\Anaconda3\envs\tf35\lib\contextlib.py", line 66, in __exit__
    next(self.gen)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Tensor must be 4-D with last dim 1, 3, or 4, not [1,3,3,4,1]
         [[Node: conv2d_1/kernel_0_1 = ImageSummary[T=DT_FLOAT, bad_color=Tensor<type: uint8 shape: [4] values: 255 0 0...>, max_images=3, _device="/job:localhost/replica:0/task:0/cpu:0"](conv2d_1/kernel_0_1/tag, ExpandDims_1)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "minist_conv.py", line 23, in <module>
    batch_size=batch_size)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\models.py", line 845, in fit
    initial_epoch=initial_epoch)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\engine\training.py", line 1485, in fit
    initial_epoch=initial_epoch)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\engine\training.py", line 1160, in _fit_loop
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\callbacks.py", line 75, in on_epoch_end
    callback.on_epoch_end(epoch, logs)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\callbacks.py", line 653, in on_epoch_end
    result = self.sess.run([self.merged], feed_dict=feed_dict)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\client\session.py", line 767, in run
    run_metadata_ptr)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\client\session.py", line 965, in _run
    feed_dict_string, options, run_metadata)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\client\session.py", line 1015, in _do_run
    target_list, options, run_metadata)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\client\session.py", line 1035, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Tensor must be 4-D with last dim 1, 3, or 4, not [1,3,3,4,1]
         [[Node: conv2d_1/kernel_0_1 = ImageSummary[T=DT_FLOAT, bad_color=Tensor<type: uint8 shape: [4] values: 255 0 0...>, max_images=3, _device="/job:localhost/replica:0/task:0/cpu:0"](conv2d_1/kernel_0_1/tag, ExpandDims_1)]]

Caused by op 'conv2d_1/kernel_0_1', defined at:
  File "minist_conv.py", line 23, in <module>
    batch_size=batch_size)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\models.py", line 845, in fit
    initial_epoch=initial_epoch)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\engine\training.py", line 1485, in fit
    initial_epoch=initial_epoch)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\engine\training.py", line 1101, in _fit_loop
    callbacks.set_model(callback_model)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\callbacks.py", line 50, in set_model
    callback.set_model(model)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\keras-2.0.2-py3.5.egg\keras\callbacks.py", line 625, in set_model
    tf.summary.image(weight.name, w_img)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\summary\summary.py", line 171, in image
    name=scope)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\ops\gen_logging_ops.py", line 200, in _image_summary
    name=name)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\framework\ops.py", line 2327, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "C:\Anaconda3\envs\tf35\lib\site-packages\tensorflow\python\framework\ops.py", line 1226, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Tensor must be 4-D with last dim 1, 3, or 4, not [1,3,3,4,1]
         [[Node: conv2d_1/kernel_0_1 = ImageSummary[T=DT_FLOAT, bad_color=Tensor<type: uint8 shape: [4] values: 255 0 0...>, max_images=3, _device="/job:localhost/replica:0/task:0/cpu:0"](conv2d_1/kernel_0_1/tag, ExpandDims_1)]]

I believe the problem lies in the dimensionality increase of the parameter tensors in callbacks.TensorBoard.set_model function.
For Conv1D, a simple workaround could be changing

w_img = tf.expand_dims(tf.expand_dims(w_img, 0), -1)

to

if len(shape) < 3:
    w_img = tf.expand_dims(tf.expand_dims(w_img, 0), -1)
elif len(shape) == 3:
    w_img = tf.expand_dims(w_img, -1)  

but in general for four-dimensional kernels this does not work.

[tg339]

• 1 Getting the same error

[tg339]

Setting use_bias=False in the Conv2D layer fixed this for me

[embanner]

@tg339 still getting the error even when supplying use_bias=False to my Conv2D layer. Could you post your working code? Perhaps you've made another change which is responsible for the the fix?

[MartinThoma]

I don't get this error with Keras 2.0.3. However, it doesn't show images either.

[QuinnLee]

What is the work around for a Conv2D?

[hanfeisun]

same issue.. Struggled on this for one whole day

[litstrong]

It was resolved by this pull request:
https://github.com/fchollet/keras/pull/6505

But if you want to support 4D weights, you need to write your own customized images.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.