Keras: Models with InputLayer are not serialized to HDF5 correctly

[cyounkins]

Opening new issue due to #10417 being closed without fix. Below is a demonstration of the issue and a hack to fix existing saved models.

docker run -it tensorflow/tensorflow:1.11.0-py3 /bin/bash
apt-get update
apt-get install python3-venv git hdf5-tools
python3 -m venv env
source env/bin/activate
pip install keras tensorflow
pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps
python test.py

import keras
from keras.models import Sequential, load_model
from keras.layers import Dense, Input, InputLayer

print('keras.__version__=', keras.__version__)

fname1 = 'test1.h5'
model1 = Sequential()
model1.add(Dense(1, input_shape=(64, 64), name='dense'))
model1.compile(loss='mse', optimizer='adam')
model1.save(fname1)
model1 = load_model(fname1)

fname2 = 'test2.h5'
model2 = Sequential()
model2.add(InputLayer((64,64), name='input'))
model2.add(Dense(1, name='dense'))
model2.compile(loss='mse', optimizer='adam')
model2.save(fname2)
# ValueError: You are trying to load a weight file containing 1 layers into a model with 0 layers
model2 = load_model(fname2)

keras.__version__= 2.2.4

$ h5dump -A test1.h5 > test1.structure
$ h5dump -A test2.h5 > test2.structure
$ diff test1.structure test2.structure
...
<       (0): "{"class_name": "Sequential", "config": {"name": "sequential_1", "layers": [{"class_name": "Dense", "config": {"units": 1, "kernel_constraint": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "use_bias": true, "dtype": "float32", "activation": "linear", "kernel_initializer": {"class_name": "VarianceScaling", "config": {"seed": null, "distribution": "uniform", "mode": "fan_avg", "scale": 1.0}}, "batch_input_shape": [null, 64, 64], "bias_constraint": null, "activity_regularizer": null, "name": "dense", "bias_regularizer": null, "trainable": true}}]}}"
---
>       (0): "{"class_name": "Sequential", "config": {"name": "sequential_2", "layers": [{"class_name": "Dense", "config": {"units": 1, "kernel_constraint": null, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "use_bias": true, "trainable": true, "activation": "linear", "kernel_initializer": {"class_name": "VarianceScaling", "config": {"seed": null, "distribution": "uniform", "mode": "fan_avg", "scale": 1.0}}, "bias_constraint": null, "activity_regularizer": null, "name": "dense", "bias_regularizer": null}}]}}"
...

The test1 has the additional structure: "batch_input_shape": [null, 64, 64], "dtype": "float32",

You can fix this using:

import json
import h5py

def fix_layer0(filename, batch_input_shape, dtype):
    with h5py.File(filename, 'r+') as f:
        model_config = json.loads(f.attrs['model_config'].decode('utf-8'))
        layer0 = model_config['config'][0]['config']
        layer0['batch_input_shape'] = batch_input_shape
        layer0['dtype'] = dtype
        f.attrs['model_config'] = json.dumps(model_config).encode('utf-8')

# Example
fix_layer0('test2.h5', [None, 64, 64], 'float32')

[ymodak]

@cyounkins Thanks for posting the workaround. Would you like to keep this issue open?

[cyounkins]

@ymodak yes absolutely. It is a bug to not be able to deserialize previously serialized networks.

[rbgreenway]

@cyounkins -- Thanks, for pushing this issue. I have been struggling with this for too long. I simply rolled back to keras 2.1.6, but would much prefer a fix...and that means fix it, not close it because one of our smarter contributors found and posted a way around the bug.

[ymodak]

@gabrieldemarmiesse Can you please take a look? Thanks!

[mikephn]

What is the workaround if my model is saved as separate .json with architecture and .hdf5 with weights?

[hosford42]

My own workaround is to build the model directly, rather than using Sequential().

[mikephn]

@hosford42 Thanks, I can construct the model in code, but the problem seems to be with loading the weight file into it...

ValueError: You are trying to load a weight file containing 1 layers into a model with 28 layers.

[hosford42]

Yes, that issue went away when I stopped using Sequential to construct the
code, and created the Input directly instead.

On Fri, Dec 21, 2018, 11:49 AM mikephn [e-mail] wrote:

@hosford42 https://github.com/hosford42 Thanks, I can construct the
model in code, but the problem seems to be with loading the weight file
into it...

ValueError: You are trying to load a weight file containing 1 layers into
a model with 28 layers.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/keras-team/keras/issues/11683#issuecomment-449454037,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEWlsxjVr9IACng1kpcS07Z02j4guuJoks5u7R82gaJpZM4Yp6NZ
.

[mikephn]

I’m using the functional API, creating the layers and calling Model(input, output) at the end.

On 21 Dec 2018, at 21:57, Aaron Hosford [e-mail] wrote:

Yes, that issue went away when I stopped using Sequential to construct the
code, and created the Input directly instead.

On Fri, Dec 21, 2018, 11:49 AM mikephn [e-mail] wrote:

@hosford42 https://github.com/hosford42 Thanks, I can construct the
model in code, but the problem seems to be with loading the weight file
into it...

ValueError: You are trying to load a weight file containing 1 layers into
a model with 28 layers.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/keras-team/keras/issues/11683#issuecomment-449454037,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEWlsxjVr9IACng1kpcS07Z02j4guuJoks5u7R82gaJpZM4Yp6NZ
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/keras-team/keras/issues/11683#issuecomment-449496441, or mute the thread https://github.com/notifications/unsubscribe-auth/AEh_HGpVtj2a-JLW91AB9fdc9NElsRy5ks5u7Us-gaJpZM4Yp6NZ.

[cottrell]

I am seeing this with class API for models saved within the same session (not cross version). Is it possible this is related?

[MmAlder]

Has anyone tried the fix (not a workaround) I proposed in #10417 ? It works for me and I haven't noticed any side effects.

[eliadl]

Another workaround is to replace:

InputLayer((64,64))

with this instead:

Lambda(tf.identity, input_shape=(64,64))