Keras-yolo3: K.sigmoid in xy_loss with binary cross entropy

[franktpmvu]

Hi,

I see the yolo head in model.py line 140

feats[...,:2] is after sigmoid function
https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/yolo3/model.py#L140

But in the loss function raw_true_xy is just inverse of line 140 without sigmoid
https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/yolo3/model.py#L380
...
In xy_loss ,raw_pred[...,0:2] also didn't have sigmoid function
https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/yolo3/model.py#L399

Why in yolo_head need sigmoid but train didn't need

Did i miss something?

Thanks!

[piantou]

your question is equal to ask "why we should employ Bounding-Box regression ", u can see this blog:https://blog.csdn.net/zijin0802034/article/details/77685438

[ancientghost]

https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/yolo3/model.py#L399

In my opinion, the above code should be

    xy_loss = object_mask * box_loss_scale * K.binary_crossentropy(raw_true_xy, K.sigmoid(raw_pred[...,0:2]), from_logits=True)

[piantou]

You know, the loss value is used to judge the difference between the real value and the predict value.
(1) Firstly, see the real value：
the predicted coordinate offset (tx, ty)，which is :
raw_true_xy = y_true[l][..., :2]*grid_shapes[l][::-1] - grid
(2) Second,see the predict value:
it is (tx,ty), which is raw_pred[...,0:2], not K.sigmoid(raw_pred[...,0:2])

so there is no sigmoid in xy_loss.

[ancientghost]

You know, the loss value is used to judge the difference between the real value and the predict value.
(1) Firstly, see the real value：
the predicted coordinate offset (tx, ty)，which is :
raw_true_xy = y_true[l][..., :2]*grid_shapes[l][::-1] - grid
(2) Second,see the predict value:
it is (tx,ty), which is raw_pred[...,0:2], not K.sigmoid(raw_pred[...,0:2])

so there is no sigmoid in xy_loss.

@piantou I dont think so. Just as you said, the loss value is used to judge the difference, therefore, the real value and predicted value should locate in the same range. Actually, the real value raw_true_xy is in [0, 1], but raw_pred[..., 0:2] not. Therefore, you need to use __sigmoid function__ to map raw_pred[..., 0:2] to [0, 1]. Of course, this operation is not necessary, but it will influence the convergence performance.

Actually, the key point is how to interpret the predicted result. Look at the following code
https://github.com/qqwweee/keras-yolo3/blob/e6598d13c703029b2686bc2eb8d5c09badf42992/yolo3/model.py#L140
It shows, sigmoid function was used at the recovery of predicted bounding boxes. That means the direct network output is not what we want instead of the output of sigmoid. Therefore, sigmoid function should be added. Of course, you can also remove both of them, but not recommended.

[franktpmvu]

When y_pred in loss function didn't have sigmoid function ,
But in the header , same variable use the sigmoid function ,
This prediction value is not equal in loss and header ,
So the detect answer is not equal to minimize losses answer we need ( loss: y_pred vs detect head: sigmoid(y_pred)) .
So the we're minimize target and same variable after header should be have some difference.
But it should not be have difference.

[ancientghost]

When y_pred in loss function didn't have sigmoid function ,
But in the header , same variable use the sigmoid function ,
This prediction value is not equal in loss and header ,
So the detect answer is not equal to minimize losses answer we need ( loss: y_pred vs detect head: sigmoid(y_pred)) .
So the we're minimize target and same variable after header should be have some difference.
But it should not be have difference.

Exactly right in my opinion, both sigmoid or both not.

[zhliu2018]

@ancientghost @franktpmvu , do you notice the args "from_logits=True"? This will do K.sigmoid(raw_pred[...,0:2])

[franktpmvu]

@ancientghost @franktpmvu , do you notice the args "from_logits=True"? This will do K.sigmoid(raw_pred[...,0:2])

oh yap ,it's finish we're problem
from_logits tags is equal to tf.softmax_cross_entropy()

in @tf_export("nn.sigmoid_cross_entropy_with_logits")

For brevity, let x = logits, z = labels. The logistic loss is
z * -log(sigmoid(x)) + (1 - z) * -log(1 - sigmoid(x))
= z * -log(1 / (1 + exp(-x))) + (1 - z) * -log(exp(-x) / (1 + exp(-x)))
= z * log(1 + exp(-x)) + (1 - z) * (-log(exp(-x)) + log(1 + exp(-x)))
= z * log(1 + exp(-x)) + (1 - z) * (x + log(1 + exp(-x))
= (1 - z) * x + log(1 + exp(-x))
= x - x * z + log(1 + exp(-x))

we're labels=z=raw_true_xy
logits=x=raw_pred[...,0:2]

thanks.

[ancientghost]

@ancientghost @franktpmvu , do you notice the args "from_logits=True"? This will do K.sigmoid(raw_pred[...,0:2])

Thanks
At first, I noticed that but didn't figure it out. Now it's clear, the original code is correct.

[Starboy-at-earth]

What is the difference between this code and its previous used square error? Does this make sense? After all, raw_true_xy is not a strict probability distribution. What is the magic behind it?

[LuTaoChen]

What is the difference between this code and its previous used square error? Does this make sense? After all, raw_true_xy is not a strict probability distribution. What is the magic behind it?

As far as I think,when we use cross entropy,which loss function is
loss=ylog(p)+(1-y)log(1-p),
which p=1/(1+exp(-z))
when take derivatives of loss with respect to z(which represent the logits), partial loss/partial z = y-p
when we use MSE 1/2*(y-p)^2, derivative of loss with respect to p, we can also get partial loss/partial p = y-p, the gradients are the same.
I guess this is why he choose to use BCE and use sigmiod to process logits to get p.Of course the final xy predction p is sigmiod(logits) in that case.