Keras: Custom objective function with external variables.

[pavitrakumar78]

Hi,

I am trying to implement a multi layer NN with a custom loss function.

Suppose I have the following function,

def custom_loss(y_true,y_pred):  

    loss = T.mean(T.square(y_pred - y_true), axis=-1)  

    return loss  

(which is just the default MSE code)
If I want to add a variable or a function of a few variables that are taken from the outside the NN - is it possible to do so?

for example:

def custom_loss(y_true,y_pred):  

    loss = T.mean(T.square(y_pred - y_true), axis=-1) + T.dot(a1,a2)  

    return loss  

where a1,a2 are defined outside.

[soloice]

Well, you can put these variables into the label y_true. This might be a hack, but it works. I don't know if there are more canonical ways.

[pavitrakumar78]

@soloice But, I thought the y_true and y_pred were passed in by the built-in functions of the Model class. Also, the function actually returns only the theano expression right? not the actual calculated value, since the function is only called once.

[lemuriandezapada]

if you use the functional api, you can simply make a new input and directly reference it in your loss.
a1 = Input(whatever_dimensions_1)
a2 = Input(whatever_dimensions_2)
and then directly call them into a loss function

[nitish11]

@lemuriandezapada : :+1:

@pavitrakumar78 :
You can also pass variable using theano shared variables, check the link below : http://deeplearning.net/software/theano/tutorial/examples.html#using-shared-variables

On the other hand, you can also try passing your defined variable tensor symbolic variable to the function.
For Example :

import theano
import theano.tensor as tensor

#definng tensor scaler 
x_input = tensor.dscalar('x') # Input
output = x_input*2  #Output

#defining theano function 
f_train = theano.function([x_input], output)

#using the defined variable
x_defined = 25 
out = f_train(x_defined)
print out

[pavitrakumar78]

@nitish11 and @lemuriandezapada Thanks for your replies!

[sharathadavanne]

so which of the methods worked for you @pavitrakumar78 ?

[pavitrakumar78]

@sharthadavanne The method @nitish11 suggested worked well for my purpose.I just passed in whatever variables I needed to the function.

[nitish11]

@sharathadavanne : try the following solution.

import theano
import theano.tensor as tensor

#definng tensor scaler 
x_input = tensor.dscalar('x') # Input
output = x_input*2  #Output

#defining theano function 
f_train = theano.function([x_input], output)

#using the defined variable
x_defined = 25 
out = f_train(x_defined)
print out
#The output will be 50.

Passing Array to tensor variable

#Numpy array
import numpy as np
x_np = np.asarray([[1,2,3],[4,5,6]])

#definng tensor scaler 
x_theano = tensor.imatrix('x') # Input for 2-D array
output = x_theano*2  # Tensor variable doing some operation on tensor matrix x_theano

#defining theano function 
f_train = theano.function([x_theano], output, allow_input_downcast=True)

#using the defined variable
out = f_train(x_np)
print out

Output will be : [[ 2 4 6] [ 8 10 12]]

For better example to shared variables and function, checkout the below code snippet.

#Giving more insights to the custom functions defined in the above cell
#The below lines are creating structure with parameters.
#You can add multiple layers of variables and functions with these parameters.

#defining tensor variables
x_scalar = tensor.dscalar('x') # Input Scalar value
x_matrix = tensor.imatrix('x') # Input for 2-D array
intermediate = x_matrix*x_scalar  # Tensor variable doing some operation on tensor matrix x_theano
function_output = intermediate*x_scalar   # Tensor variable doing some operation on tensor matrix intermediate

#This is a theano function definition which will take list of theano variables as 'input' defined above and do the operations and return the output in the 'function_output' variable.
f_train = theano.function([x_matrix, x_scalar], function_output, allow_input_downcast=True, on_unused_input='ignore')

#This is the function call which will take the arguments and execute the function and return the output.
x_val = 2
x_np = np.asarray([[1,2,3],[4,5,6]])
out = f_train(x_np, x_val)
print out

Output will be : [[ 4. 8. 12.] [ 16. 20. 24.]]