Keras: LSTM: 'You must feed a value for placeholder tensor' error.

[fkluger]

• [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps

• [x] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• [ ] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

• [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

The following code works fine with a GPU installation of Keras w/ Tensorflow, but raises an error when using a CPU-only installation. Said error only occurs when I use an LSTM cell with return_state=True and initial_state set within a model, and then use that model as a Layer within another model.
I successfully trained and tested a model with a similar structure on an Nvidia GPU, but am unable to deploy it on my laptop (w/ Intel GPU only) due to this error.

from keras.models import Model
from keras.layers import Input, LSTM
import numpy as np

def model1():
    input_state = Input(shape=(1,))
    input_data = Input(shape=(1, 1))

    output, _, _ = LSTM(1, return_state=True)(input_data, initial_state=[input_state, input_state])

    return Model(inputs=[input_state, input_data], outputs=[output])

def model2():
    input_state = Input(shape=(1,))
    input_data = Input(shape=(1, 1))

    # Using model1 as a layer raises an error:
    output = model1()([input_state, input_data])
    # Using the following line instead gives no error:
    # output, _, _ = LSTM(1, return_state=True)(input_data, initial_state=[input_state, input_state])

    return Model(inputs=[input_state, input_data], outputs=[output])


model = model2()

p = model.predict([np.zeros((1,1)), np.zeros((1,1,1))])

print(p)

This gives me:

/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
2018-01-19 00:42:26.119981: I tensorflow/core/platform/s3/aws_logging.cc:53] Initializing Curl library
Traceback (most recent call last):
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1350, in _do_call
    return fn(*args)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1329, in _run_fn
    status, run_metadata)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input_3' with dtype float and shape [?,1]
     [[Node: input_3 = Placeholder[dtype=DT_FLOAT, shape=[?,1], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/kluger/projects/test_example/test.py", line 33, in <module>
    p = model.predict([np.zeros((1,1)), np.zeros((1,1,1))])
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/engine/training.py", line 1800, in predict
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/engine/training.py", line 1301, in _predict_loop
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/backend/tensorflow_backend.py", line 2475, in __call__
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1128, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
    options, run_metadata)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input_3' with dtype float and shape [?,1]
     [[Node: input_3 = Placeholder[dtype=DT_FLOAT, shape=[?,1], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op 'input_3', defined at:
  File "/home/kluger/projects/test_example/test.py", line 31, in <module>
    model = model2()
  File "/home/kluger/projects/test_example/test.py", line 23, in model2
    output = model1()([input_state, input_data])
  File "/home/kluger/projects/test_example/test.py", line 8, in model1
    input_state = Input(shape=(1,))
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/engine/topology.py", line 1455, in Input
    input_tensor=tensor)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/engine/topology.py", line 1364, in __init__
    name=self.name)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/backend/tensorflow_backend.py", line 504, in placeholder
    x = tf.placeholder(dtype, shape=shape, name=name)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1717, in placeholder
    return gen_array_ops._placeholder(dtype=dtype, shape=shape, name=name)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3051, in _placeholder
    "Placeholder", dtype=dtype, shape=shape, name=name)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3172, in create_op
    op_def=op_def)
  File "/home/kluger/.anaconda/envs/keras/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1617, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input_3' with dtype float and shape [?,1]
     [[Node: input_3 = Placeholder[dtype=DT_FLOAT, shape=[?,1], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

[pavithrasv]

I see the error both on CPU and GPU installation. The 'input_3' which is missing a value is input_state that you have created inside of model1(). 3 is because it is the third input layer getting invoked in the code flow. Btw, if you add names to layers, it will help understand error messages better: input_state = Input(shape=(1,), name='lstm_input_state').

When you call the LSTM layer you have created a dependency to input_3 tensor but the input state that is being passed is value for input_1. You can fix this by making sure the input state you are passing comes from original input or by using the LSTM layer directly in model2[as you have done in the line you have commented out]

[fkluger]

For me, the same code happily returns [[0.]] on another machine:

~/projects/test_example $ python test.py 
Using TensorFlow backend.
2018-01-19 09:02:27.037962: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-01-19 09:02:27.038000: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-01-19 09:02:27.038005: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-01-19 09:02:27.038010: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-01-19 09:02:27.038013: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2018-01-19 09:02:27.479888: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties: 
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.582
pciBusID 0000:84:00.0
Total memory: 10.91GiB
Free memory: 4.56GiB
2018-01-19 09:02:27.479949: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 
2018-01-19 09:02:27.479965: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y 
2018-01-19 09:02:27.479986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:84:00.0)
[[ 0.]]

On that machine, Keras/Tensoflow is not quite up to date, so it could be just a version issue (instead of CPU vs. GPU) after all:

$ conda list
...
keras-gpu                 2.0.8            py36h0585f72_0  
...
tensorflow-gpu            1.3.0                         0  
tensorflow-gpu-base       1.3.0           py36cuda8.0cudnn6.0_1  
tensorflow-tensorboard    0.1.5                    py36_0  
...

As I said, I successfully trained a model with a similar structure on that machine, but cannot deploy it on another one.

Edit: I downgraded to Keras 2.0.8 and Tensorflow 1.3.0 on the CPU-only machine and now the code works! This is a suitable workaround for me for now, but will this bug be fixed in the future? The model I defined in the first post should be perfectly valid IMO.

[cpury]

I'm getting the same error in Keras 2.2.0 and TensorFlow 1.8.0, but not when creating the model, but when running it with a TensorBoard callback with histogram_freq=1 at the end of the first epoch:

InvalidArgumentError: You must feed a value for placeholder tensor 'dense_6_sample_weights' with dtype float and shape [?]
         [[Node: dense_6_sample_weights = Placeholder[dtype=DT_FLOAT, shape=[?], _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

[Corrigenda]

I am also getting this error from the TensorBoard callback. Removing the histogram_freq = 1 value seems to get rid of it.