Keras: Invalid activation function

[diogoff]

I'm training a model using a custom activation (theano) function. When training is finished, I save the model with f.write(model.to_json()).

In another Python script, I load the model with model = model_from_json(f.read()) but I get the following exception:

Traceback (most recent call last):
model = model_from_json(f.read())
File "/usr/lib/python2.7/site-packages/keras/models.py", line 139, in model_from_json
return model_from_config(config, custom_objects=custom_objects)
File "/usr/lib/python2.7/site-packages/keras/models.py", line 150, in model_from_config
model = container_from_config(config, custom_objects=custom_objects)
File "/usr/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 57, in container_from_config
layer = container_from_config(layer_dict['nodes'].get(node['name']))
File "/usr/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 92, in container_from_config
base_layer = get_layer(name, layer_dict)
File "/usr/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 158, in get_layer
instantiate=True, kwargs=kwargs)
File "/usr/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 18, in get_from_module
return res(kwargs)
File "/usr/lib/python2.7/site-packages/keras/layers/core.py", line 668, in **init
self.activation = activations.get(activation)
File "/usr/lib/python2.7/site-packages/keras/activations.py", line 51, in get
return get_from_module(identifier, globals(), 'activation function')
File "/usr/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 14, in get_from_module
str(identifier))
Exception: Invalid activation function: (my custom activation function)

I tried re-declaring the function in the second script but that didn't work either.

So how can I get the second script (the one that loads the model) to recognize the custom activation function that I used in the first script (the one that saved the model)? Thanks.

[tboquet]

Try to pass your custom activation via the custom_objects parameter. You should provide a dict:

model = model_from_json(f.read(), custom_objects={'my_custom_activation': my_custom_activation})

If you want more details, you could find the code here:
https://github.com/fchollet/keras/blob/master/keras/models.py#L162

[diogoff]

Thanks!

[danjgale]

Hi,

I am receiving the exact same error with an identical situation. However, even after specifying the custom activation function in the custom_objects argument, I still receive the same error. Any ideas?

Thanks!

[paul-felt]

I'm seeing the same error. It looks like the error happens when utils.layer_utils.layer_from_config sets the custom objects in globals() assuming that they will be available to all downstream code. However, globals() scoped at a module level (http://stackoverflow.com/questions/15959534/python-visibility-of-global-variables-in-imported-modules), so invocations to other modules (e.g., engine/topology) lose access to the values that were set.

[stergioc]

Hi,

I just bumped on the same error. Here is a sample code that reproduces the problem.

from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.optimizers import SGD
from keras import backend as K 
from keras.models import model_from_json

# custom softmax
def custom(x):
    e = K.exp(x)
    s = K.sum(e, axis=-1, keepdims=True)
    return e / s

# create a model with the custom softmax
model = Sequential()
model.add(Dense(output_dim=64, input_dim=100))
model.add(Activation("relu"))
model.add(Dense(output_dim=10))
model.add(Activation(custom))

# compile the model 
model.compile(loss='categorical_crossentropy', optimizer='sgd')
print "Compilation OK!"

# load the model from json string
json_string = model.to_json()
model = model_from_json(json_string, custom_objects={'custom':custom})
print "Load OK!"

[armancohan]

Experiencing the same issue. Any updates on this?

[JasonKessler]

Ditto for me. However, I came up with a horrendous work around, which is basically just to patch the activations.get function with your custom function. It works more or less like:

def custom(x): return x

old_get = keras.activations.get
def patch_get(x): 
    return custom if x == 'custom' else old_get(x)
@mock.patch('keras.activations.get', patch_get)
def load():
    with open('config.json') as file_:
        model = model_from_json(file_.read(), custom_objects={'custom': custom})
    ...

[PeterChe1990]

Thanks JasonKessler! Actually I've been watching this problem for a while and this is the best solution I can find so far, across github, google group, and google search. I'd like to mention that this problem is not only limited to activation but also weight regularizer, constraint, and initialization - basically everything that a layer needs to call get() to create in the layer's own __init__() method. Hope Keras will fix this soon..

[JasonKessler]

Happy to help, @PeterChe1990 !

[thouis]

Another (maybe simpler?) way to work around this is to add your custom activation to the keras.activations module:

import keras.activations
keras.activations.custom_activation = custom_activation

[joeyearsley]

https://github.com/fchollet/keras/pull/5012

from keras.utils.generic_utils import get_custom_objects  
metrics = CustomLosses()  
get_custom_objects().update({"dice_coef_class_loss": metrics.dice_coef_class_loss})