Keras: Errors when evaluating a tensor in custom loss function?

[oxrider]

I've been trying to the shape of a tensor in the custom loss function but Keras has been giving me this error at model.compile(loss=custom_loss, ...):

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'dense_4_target' with dtype float
[[Node: dense_4_target = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

and here is my code for it:

def custom_loss(y_true, y_pred):
    print "Reached"
    print K.eval(K.shape(y_true))
    print "Not reached"

I understand that a Tensor is a virtual place holder that only has values (and shapes) when it is filled, but shouldn't it be filled when the loss function is called? Thank you very much for your help!

[JimBiardCics]

I think perhaps the problem here is that when the function is called the arguments aren't filled yet. I believe the loss function is supposed to return a compiled Tensor function that will be evaluated later.

[patrick-ogrady]

is there a fix?

[jaekyeom]

Any workarounds?

[Pezaun]

I have the same issue. I have to access data of the y_pred into my custom loss function. I agree with JimBiardCics, the tensor mus be filled first, but I don't know how to deal with this issue. :/

[JimBiardCics]

Keep in mind that the python function you write (custom_loss) is called to generate and compile a C function. The compiled function is what is called during training. When your python custom_loss function is called, the arguments are tensor objects that don't have data attached to them. The K.eval call will fail, as will the K.shape call. The only thing you can really know about your arguments is the number of dimensions. You must write your function in such a fashion that it deals with things in a more symbolic fashion. Look at the source for the different loss functions provided by Keras for inspiration.

[Pezaun]

So, that's it! I have to compute bounding boxes from my y_pred, to compare it with ground truth. My python code converts from a array of 1470 values to n bounding boxes. I guess that my custom loss can not call this python function. :/

So, I'm lost! :/

[jaekyeom]

@Pezaun How about trying rewriting your function using symbolic operations from keras.backend?

[Pezaun]

It is exactly what I'm looking for. However, I'm not sure if it's possible to implement the approximation trough only backend operations. :/ My convolutional output has 1470 features that represents 98 bounding boxes. I have to reconstruct the boxes and validate it across the ground truth. Basically it is part of YOLO (http://pjreddie.com/darknet/yolo/) object segmentation approach.

[JimBiardCics]

Checkout the Theano or TensorFlow (depending on which backend you are
using). They both have a rich set of tensor functions which can
accomplish a lot. I had to learn Theano in order to some of the things
I'm doing with Keras. Here are a couple of links useful parts of the
Theano documentation.

http://deeplearning.net/software/theano/tutorial/index.html#

http://deeplearning.net/software/theano/library/tensor/index.html

[Pezaun]

Thanks for your help! I'm using tensorflow and I will look for tensor functions to implement the loss.

[mdalvi]

@Pezaun I am having the same issue, @ Converting Tensor to np.array using K.eval() in Keras returns InvalidArgumentError

Did you found the solution?

[alyato]

hi.Thanks for your guys do this. But i think this question can't be solved.
@JimBiardCics Do you share one example to explain it?
Thanks again.

[JimBiardCics]

@alyato There was/is no bug here. The problem is a lack of understanding. When you write a custom loss function, you are writing a function that generates a function. Your function is called before any data is prepared. The arguments to your function are placeholder objects, not actual data arrays. As an example, here is a custom loss function I wrote that applies class weights. (It's ugly and depends on a global, but it shows what's going on.)

def categoricalCrossentropy(y_true, y_pred):
    '''
    Calculate the class-weighted categorical cross-entropy for the given
    predicted and true sets.

    y_true [in] The truth set to test against. This is a Tensor with a last
                dimension that contains a set of 1-of-N selections.
    y_pred [in] The predicted set to test against. This is a Tensor with a last
                dimension that contains a set of 1-of-N selections.
    returns     A Tensor function that will calculate the weighted categorical
                cross-entropy on the inputs.
    '''

    # If weights are defined, multiply the truth values by the class weights.
    #
    if __lossWeights is not None:
        # Wrap the loss weights in a tensor object.
        #
        theWeights = backend.constant(__lossWeights, shape = __lossWeights.shape)

        y_true *= theWeights

    # Get the cross-entropy and return it.
    #
    crossEntropy = backend.categorical_crossentropy(y_true, y_pred)

    return crossEntropy

Keras only calls this function once, while compiling the model. It appears that a data array is being returned, but it is actually a tensor function that will be called to do the actual calculation while the model is being run. Each statement in this function is, in essence, being recorded and used to build the function that will be called.

Here is another example. I used the Keras backend to write an application that calculates a bunch of metrics on some truth and predicted data. Here's the function that created a custom categorical cross-entropy function that allowed for a spatial mask (locations where the truth data had no class selected).

def create_categorical_crossentropy():
    '''
    Create the categorical cross-entropy function.

    returns     A Tensor function that will calculate the categorical
                cross-entropy on the inputs.
    '''

    # Create the truth and predicted output tensors.
    #
    pred  = backend.placeholder(ndim = 4, dtype = 'float32', name = 'pred')
    truth = backend.placeholder(ndim = 4, dtype = 'float32', name = 'truth')

    # Clip zeros to 1.0e-7 to avoid numerical instability.
    #
    pred2 = backend.clip(pred, 1.0e-7, 1.0)

    # Get the element-wise categorical cross-entropy tensor, then get the sum
    # of all the elements.
    #
    crossentropy = backend.categorical_crossentropy(truth, pred2)
    crossentropy = backend.sum(crossentropy)

    # Get the number of valid (non-zero) truth entries.
    #
    mask = backend.any(truth, axis = -1, keepdims = True)

    valid = backend.sum(mask)

    # Get the mean cross-entropy value.
    #
    crossentropy /= backend.cast(valid, 'float32')

    # Return the Tensor function that will calculate the cross-entropy.
    #
    return backend.function((truth, pred), crossentropy)

Here's how I called the function.

lossFunc  = create_categorical_crossentropy()

And here's how I used the generated loss function.

theLoss = lossFunc((truthArray, predArray))

The arguments truthArray and predArray are numpy arrays.

Notice that the first function has no arguments. This is an extreme example, but I wrote it for a specific case where I knew the dimensionality of my inputs. The last statement in the function causes a theano or tensorflow function to be created that will perform the calculations called out by the previous six statements. The statements creating the tensor placeholders and creating the theano or tensorflow function are normally handled by Keras during model compilation.

I call create_categorical_crossentropy in order to get the function that will do the actual calculation. I call the generated function with actual arguments and get an actual result.