Keras: Keras Applications + TimeDistributed Error

[thierrypin]

Hello, I found some strange behavior in Keras. I'm trying to classify a video by describing each frame with MobileNet in a TimeDistributed way and applying GRU layers on the output. But whenever I apply a pre-trained model inside a TimeDistributed wrapper, most of the outputs are dropped to 0.

Here is a sample:

import numpy as np
from keras.applications.mobilenet import MobileNet
from keras.models import load_model, Model
from keras.layers import Input
from keras.layers import TimeDistributed

# Get some preprocessed video
crop = np.load(path_to_npy) # shape: 50, 64, 64, 3
# Apply mobilenet description
mobilenet = MobileNet(include_top=False, weights='imagenet', pooling='max')
# get the features
feats_1 = mobilenet.predict(crop)

# Pass it to a model
input_layer = Input(shape=(50,64,64,3))
features = TimeDistributed(mobilenet)(input_layer)
model = Model(inputs=input_layer, outputs=features)
# ... and the new features are different!
feats_2 = model.predict(crop[np.newaxis, ...])

print("feats_1:\n", feats_1)
print()
print("feats_2:\n", feats_2)

Output:

feats_1:
 [[ -6.225078     0.5843177   -2.2938356  ...  -0.89581835  -3.5828576
   -2.258006  ]
 [ -7.3269567    1.0817679   -2.1962285  ...  -1.4383472   -3.6848595
   -3.1082852 ]
 [ -7.572564     1.7693791   -3.9287426  ...  -1.4699366   -3.3960166
   -1.3519664 ]
 ...
 [-11.804419     1.6854403   -6.3973193  ...  -2.3420107   -6.269591
   -2.7979271 ]
 [ -7.309808     1.2738572   -4.458527   ...  -3.3302228   -6.9717727
   -6.512269  ]
 [ -8.261859     0.8793758   -3.0096955  ...  -1.3412896   -3.0525224
   -3.7835257 ]]

feats_2:
 [[[0.         0.5843177  0.         ... 0.         0.         0.        ]
  [0.         1.0817679  0.         ... 0.         0.         0.        ]
  [0.         1.7693791  0.         ... 0.         0.         0.        ]
  ...
  [0.         0.         0.         ... 0.         0.5828055  0.        ]
  [0.         0.         0.         ... 0.         0.         0.        ]
  [0.         0.         0.         ... 0.67905384 1.331172   0.        ]]]

Notice that some elements are dropped to zero, but the ones that aren't have the right value.

• [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps

• [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

• [x] Check that your version of TensorFlow is up-to-date. The installation instructions can be found here.

[see--]

@jvishnuvardhan Isn't this a possible bug? I'd expect both outputs to be the same, too. Here is a self-contained example:

from keras.applications.mobilenet import MobileNet
from keras.models import Model
from keras.layers import Input
from keras.layers import TimeDistributed
import numpy as np

np.random.seed(42)
# Get some preprocessed video
crop = np.random.rand(2, 64, 64, 3) * 0.5 - 0.5
# Apply mobilenet description
mobilenet = MobileNet(include_top=False, weights='imagenet', pooling='max')
# get the features
feats_1 = mobilenet.predict(crop)

# Pass it to a model
input_layer = Input(shape=(2, 64, 64, 3))
features = TimeDistributed(mobilenet)(input_layer)
model = Model(inputs=input_layer, outputs=features)
# ... and the new features are different!
feats_2 = model.predict(crop[np.newaxis, ...])

print("feats_1:\n", feats_1)
print()
print("feats_2:\n", feats_2)

Output:

feats_1:
 [[  0.34682608 -10.600048    -5.786285   ...  -6.7834044   -9.104498
   -5.8894067 ]
 [ -2.4729424   -7.7848043   -6.2087708  ... -10.531845    -9.776144
   -0.57902145]]

feats_2:
 [[[0.34682608 0.         0.         ... 0.         0.         0.        ]
  [0.         0.         0.         ... 0.         0.         0.        ]]]

[jvishnuvardhan]

@thierrypin It looks like a bug. Could you create a code to reproduce the bug? Please use any public datasets (private is fine too) to show the bug. Thanks!