Keras: Loss function for tensor from multi-output layer

[agamemnonc]

I have a somewhat complicated cost function which includes terms for auxiliary outputs. Some of the auxiliary outputs come from layers with multiple output nodes (because the same layers are used multiple times in the pipeline).

So far, so good. Now my question is, Is there a way to specify which output nodes (i.e. specific tensors) my loss function will be applied to?

[Dref360]

As stated in the docs: https://keras.io/models/model/

You can specify a dict :
loss = {'name_of_output1':'mse'}

[agamemnonc]

Thanks, I am aware of this.

In your example name_of_output1 is a layer, not a tensor. If the layer has a single output node, that's no problem.

However, in the case where the layer has multiple output nodes, is there a way of specifying which output node (i.e. tensor) the loss function is applied to?

[rex-yue-wu]

Yes, one trick is to use Activation('linear') layer to produce a dummy identical output. For example, you can do

vgg = VGG16()
x1 = Input( shape=(224,224,3), name='x1' )
x2 = Input( shape=(224,224,3), name='x2' )
y1 = vgg(x1)
y2 = vgg(x2)
z1 = Activation('linear', name='z1')(y1)
z2 = Activation('linear', name='z2')(y2)
mom = Model( inputs=[x1,x2], outputs=[z1,z2], name='multiOutputModel' )
mom.compile( 'sgd', loss = {'z1' : 'mse', 'z2' : 'catergorical_crossentropy' } )

[Dref360]

Oh Sorry, I read too fast :(
You could wrap your layer inside Lambda maybe?

layerN = ... # the layer with mutliple outputs
output1 = Lambda(lambda l: l.outputs[0])(layerN)
output2 = Lambda(lambda l: l.outputs[1])(layerN)

This will keep the _keras_shape.

[agamemnonc]

Thanks, both workarounds make sense.

Please feel free to close this issue, unless you reckon that this is a feature worth (and easy) supporting out of the box.

[Dref360]

Have you any proposition? That all outputs are wrapped in a Keras layer? (basically adding 2 fields)

Do you have a list of all Layers who return multiple outputs?

[fchollet]

Yes, the Lambda workaround is the best. Basically if you want distinct names for your outputs, they need to come from distinct layers.

[agamemnonc]

Thanks, in that case (i.e. distinct layers) how is it possible to force them to share their parameters?

[fchollet]

As stated, Activation or Lambda is fine to get a new endpoint name for a data flow.

[thebeancounter]

Yes, one trick is to use Activation('linear') layer to produce a dummy identical output. For example, you can do

vgg = VGG16()
x1 = Input( shape=(224,224,3), name='x1' )
x2 = Input( shape=(224,224,3), name='x2' )
y1 = vgg(x1)
y2 = vgg(x2)
z1 = Activation('linear', name='z1')(y1)
z2 = Activation('linear', name='z2')(y2)
mom = Model( inputs=[x1,x2], outputs=[z1,z2], name='multiOutputModel' )
mom.compile( 'sgd', loss = {'z1' : 'mse', 'z2' : 'catergorical_crossentropy' } )

could I use this with my custom loss functions?

let's say that i want to implement triplet loss for a network with 3 outputs: z1, z2, z3 and have a function that recieved 3 vectors in order to calculate the loss

def loss(i1, i2, i3):
    return something_with(i1,i2,i3)