Keras: PReLU layer not working with variable input size

[StefanL19]

I want to implement a sample PNet model from this paper. For this purpose I am using the functional API instead of the Sequential model, since at the end I need to concatenate two outputs.

This is the code:

from  keras import backend as K
from keras.models import Sequential, Model, Input
from keras.layers import Conv2D, MaxPooling2D, Dense, Activation
from keras.layers.advanced_activations import PReLU
K.set_image_dim_ordering('tf')
def get_p_net():
    main_input = Input(shape=(None, None, 3), dtype='float32', name='main_input')
    x = Conv2D(10, (3, 3), strides=(1, 1), padding='valid', name='conv1')(main_input)
    x = PReLU(name='PReLU1', alpha_constraint=None)(x)
    x = MaxPooling2D(pool_size=(2, 2), strides=(2,2), padding='same', name='pool1')(x)
    x = Conv2D(16, (3, 3), strides=(1,1), padding='valid', name='conv2')(x)
    x = PReLU(name='PReLU2')(x)
    x = Conv2D(32, (3, 3), strides=(1,1), padding='valid', name='conv3')(x)
    x = PReLU(name='PReLU3')(x)
    binary_face_output = Conv2D(2, (1, 1), strides=(1,1), padding='same', name='conv4-1')(x)
    binary_face_output = Dense(2, activation='softmax', name='prob1')(binary_face_output)
    bbox_output = Conv2D(4, (1, 1), strides=(1,1), padding='same', name='conv4-2')(x)
    model = Model(inputs=[main_input], outputs=[binary_face_output, bbox_output])
    model.summary()
    # model.compile(optimizer='rmsprop', loss='binary_crossentropy',
    #           loss_weights=[1., 1.])

get_p_net()

Everything looks ok, except when I try running the code I receive the following error.

```Traceback (most recent call last):
File "p_net.py", line 23, in
get_p_net()
File "p_net.py", line 9, in get_p_net
x = PReLU(name='PReLU1', alpha_constraint=None)(x)
File "D:\Anaconda2envs\tensorflow\lib\site-packages\kerasengine\topology.py", line 569, in __call__
self.build(input_shapes[0])
File "D:\Anaconda2envs\tensorflow\lib\site-packages\keras\layers\advanced_activations.py", line 111, in build
constraint=self.alpha_constraint)
File "D:\Anaconda2envs\tensorflow\lib\site-packages\keras\legacy\interfaces.py", line 87, in wrapper
return func(args, *kwargs)
File "D:\Anaconda2envs\tensorflow\lib\site-packages\kerasengine\topology.py", line 391, in add_weight
weight = K.variable(initializer(shape), dtype=dtype, name=name)
File "D:\Anaconda2envs\tensorflow\lib\site-packages\keras\initializers.py", line 29, in __call__
return K.constant(0, shape=shape, dtype=dtype)
File "D:\Anaconda2envs\tensorflow\lib\site-packages\keras\backend\tensorflow_backend.py", line 358, in constant
return tf.constant(value, dtype=dtype, shape=shape, name=name)
File "D:\Anaconda2envs\tensorflow\lib\site-packages\tensorflow\python\framework\constant_op.py", line 102, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "D:\Anaconda2envs\tensorflow\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 371, in make_tensor_proto
if np.prod(shape) == 0:
File "D:\Anaconda2envs\tensorflow\lib\site-packages\numpy\core\fromnumeric.py", line 2518, in prod
out=out, **kwargs)
File "D:\Anaconda2envs\tensorflow\lib\site-packages\numpy\core_methods.py", line 35, in _prod
return umr_prod(a, axis, dtype, out, keepdims)
TypeError: unsupported operand type(s) for *: 'NoneType' and 'NoneType'

Fixed the problem and I get the model summary. I want to ask if there is a way of processing a variable size inputs through a network which contains a `PReLU` layers.
It is clear that somehow it does not like the input shape of the layer (output of the last convolutional layer). So, changing the very first line to:

main_input = Input(shape=(12, 12, 3), dtype='float32', name='main_input')
```

I want to ask if there is a way of processing variable size inputs through a network which contains PReLU layers.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[Wuppus]

Same problem here - I want to use variable size inputs with PReLU. I saw a similar issue dated earlier this year with also no response.

[avinres]

I am also getting the same problem. Don't know how to solve

[singhay]

This is a bug

• ✅ Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps

• ✅ If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

Related Issue: https://github.com/keras-team/keras/issues/7694

The bug most likely starts in Line 109 build() function of PReLU class of file advanced_activations.py line

        param_shape = list(input_shape[1:])

which assumes that one the first position will contain None representing batch size, however for a greater tensor that has None at index 1 will let this being passed further resulting in TypeError

[Xonxt]

Any updates on that? I'm currently using version 1.13.1 of tensorflow and keras 2.2.4.
The problem is still there.

It can be sorta solved, if you put PReLU(shared_axes=[1,2]), but I'm not sure if that's a proper solution.

error log:

---> 15 x = PReLU()(x)
     16 

c:\program files\python36\lib\site-packages\keras\engine\base_layer.py in __call__(self, inputs, **kwargs)
    429                                          'You can build it manually via: '
    430                                          '`layer.build(batch_input_shape)`')
--> 431                 self.build(unpack_singleton(input_shapes))
    432                 self.built = True
    433 

c:\program files\python36\lib\site-packages\keras\layers\advanced_activations.py in build(self, input_shape)
    118                                      initializer=self.alpha_initializer,
    119                                      regularizer=self.alpha_regularizer,
--> 120                                      constraint=self.alpha_constraint)
    121         # Set input spec
    122         axes = {}

c:\program files\python36\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

c:\program files\python36\lib\site-packages\keras\engine\base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint)
    247         if dtype is None:
    248             dtype = K.floatx()
--> 249         weight = K.variable(initializer(shape),
    250                             dtype=dtype,
    251                             name=name,

c:\program files\python36\lib\site-packages\keras\initializers.py in __call__(self, shape, dtype)
     36 
     37     def __call__(self, shape, dtype=None):
---> 38         return K.constant(0, shape=shape, dtype=dtype)
     39 
     40 

c:\program files\python36\lib\site-packages\keras\backend\tensorflow_backend.py in constant(value, dtype, shape, name)
    428     if dtype is None:
    429         dtype = floatx()
--> 430     return tf.constant(value, dtype=dtype, shape=shape, name=name)
    431 
    432 

c:\program files\python36\lib\site-packages\tensorflow\python\framework\constant_op.py in constant_v1(value, dtype, shape, name, verify_shape)
    177   """
    178   return _constant_impl(value, dtype, shape, name, verify_shape=verify_shape,
--> 179                         allow_broadcast=False)
    180 
    181 

c:\program files\python36\lib\site-packages\tensorflow\python\framework\constant_op.py in _constant_impl(value, dtype, shape, name, verify_shape, allow_broadcast)
    281       tensor_util.make_tensor_proto(
    282           value, dtype=dtype, shape=shape, verify_shape=verify_shape,
--> 283           allow_broadcast=allow_broadcast))
    284   dtype_value = attr_value_pb2.AttrValue(type=tensor_value.tensor.dtype)
    285   const_tensor = g.create_op(

c:\program files\python36\lib\site-packages\tensorflow\python\framework\tensor_util.py in make_tensor_proto(values, dtype, shape, verify_shape, allow_broadcast)
    461     # If shape is None, numpy.prod returns None when dtype is not set, but raises
    462     # exception when dtype is set to np.int64
--> 463     if shape is not None and np.prod(shape, dtype=np.int64) == 0:
    464       nparray = np.empty(shape, dtype=np_dt)
    465     else:

c:\program files\python36\lib\site-packages\numpy\core\fromnumeric.py in prod(a, axis, dtype, out, keepdims, initial)
   2770     """
   2771     return _wrapreduction(a, np.multiply, 'prod', axis, dtype, out, keepdims=keepdims,
-> 2772                           initial=initial)
   2773 
   2774 

c:\program files\python36\lib\site-packages\numpy\core\fromnumeric.py in _wrapreduction(obj, ufunc, method, axis, dtype, out, **kwargs)
     84                 return reduction(axis=axis, out=out, **passkwargs)
     85 
---> 86     return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
     87 
     88 

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

[zaccharieramzi]

Roughly the same stack trace as @Xonxt .