Keras: TypeError: unsupported operand type(s) for +: 'NoneType' and 'int' in new version Tensorflow backend

[frozenxia]

hello, I have a define a model as below,

def cnn_multi_filters(sentence_length, embeddings, class_num=3, kernel_filters=[3, 4, 5]):
    input_text = Input(shape=(sentence_length,), dtype='int32')
    emb_text = embedding_layers.embeddings_layer(embeddings, sentence_length)(input_text)
    gaussian_noise = 0
    drop_text_input_rate = 0.2
    emb_text = GaussianNoise(gaussian_noise)(emb_text)
    emb_text = Dropout(rate=drop_text_input_rate)(emb_text)

    pooling_reps = []
    nfilters = 64
    drop_conv = 0.3
    for kf in kernel_filters:
        feat_maps = Conv1D(filters=nfilters, kernel_size=kf, activation='relu')(emb_text)
        pool_vecs = MaxPooling1D(pool_size=2)(feat_maps)
        pool_vecs = Flatten()(pool_vecs)
        print(pool_vecs)
        pooling_reps.append(pool_vecs)

    representation1 = concatenate(pooling_reps)
    print(representation1)
    representation2 = Dropout(drop_conv)(representation1)
    print(representation2)


    probablities = Dense(units=class_num, name='probability')(representation2)

    results = Activation(activation='softmax')(probablities)
    model = Model(inputs=input_text, outputs=results)
    model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

    model_to_intermedia = Model(inputs=input_text, outputs=probablities)
    return model, model_to_intermedia

it works fine on Keras 2.2.4 with tensorflow(1.10.0) as backend. But when i run it on Keras 2.2.4 with tensorflow(1.12.0), it throws exception as bellow:

Traceback (most recent call last):
  File "/home/mi/github/gitv9/text-classification-models/runModel.py", line 13, in <module>
    fastCNN.cnn_multi_filters(sentence_length=240,embeddings=(vocab_size,embedding_size))
  File "/home/mi/github/gitv9/text-classification-models/tc/models/fastCNN.py", line 86, in cnn_multi_filters
    probablities = Dense(units=class_num, name='probability')(representation2)
  File "/home/mi/anaconda3/lib/python3.6/site-packages/keras/engine/base_layer.py", line 431, in __call__
    self.build(unpack_singleton(input_shapes))
  File "/home/mi/anaconda3/lib/python3.6/site-packages/keras/layers/core.py", line 866, in build
    constraint=self.kernel_constraint)
  File "/home/mi/anaconda3/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/mi/anaconda3/lib/python3.6/site-packages/keras/engine/base_layer.py", line 249, in add_weight
    weight = K.variable(initializer(shape),
  File "/home/mi/anaconda3/lib/python3.6/site-packages/keras/initializers.py", line 209, in __call__
    scale /= max(1., float(fan_in + fan_out) / 2)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

I wonder where is the problem, hope someone can give some clue.

[gabrieldemarmiesse]

Thanks for the report! Can you reduce the size of your example as much as possible while keeping the bug? This will help making the debugging faster.

[frozenxia]

@gabrieldemarmiesse , I have simplify the model as bellows, it still have type error. Hope this can make it easier for you to debug

    input_text = Input(shape=(240,), dtype='int32')
    emb_text = Embedding(
                input_dim = 16000,
                output_dim = 64,
                input_length = 240 ,
                trainable = True,
                name = 'layer_embedding'
                )(input_text)
    pooling_reps = []
    for kf in (3,4,5):
        feat_maps = Conv1D(filters=64, kernel_size=kf, activation='relu')(emb_text)
        pool_vecs = Flatten()(feat_maps)
        pooling_reps.append(pool_vecs)
    representation1 = concatenate(pooling_reps)
    probablities = Dense(units=class_num, name='probability')(representation1)
    results = Activation(activation='softmax')(probablities)
    model = Model(inputs=input_text, outputs=results)
    return model

the error occurs at probablities = Dense(units=class_num, name='probability')(representation1)

[gabrieldemarmiesse]

Thanks for the script!

[PatakiMarton]

I had the same problem while constructing RNN networks, changing the input shape from (n, ) to (1 , n, 1) solved it for me but I just tested it with model.summary() I will try and change "1" to specific values later and will update.