Dlib: simple_object_detector probabilities

Created on 9 Sep 2020 · 9Comments · Source: davisking/dlib

Hello,

Does simple_object_detector return probabilities? While validating/testing it would be nice to get probabilities corresponding to the boxes. I can't seem to find such probabilities, however.

All in all, the detector is very nice. Thank you for providing such a tool.

inactive

Source

liskieero

All 9 comments

It does not. It outputs a real value that is bigger when something is more likely to be a real object. It's the output of the linear filter that detects the object. If you want a probability you should scale that number into the range 0/1. A good way to do this is to use Platt scaling: http://dlib.net/ml.html#learn_platt_scaling

davisking on 10 Sep 2020

I sometimes use these two functions to go from the output of the filter to something that _looks like_ a probability, but was never satisfied, sometimes it's over/under-confident...

double sigmoid(const double val, const double alpha = 1.0)
{
    return 1.0 / (1.0 + std::exp(-alpha * val));
}

double logit(const double val, const double alpha = 1.0)
{
    return 1.0 / alpha * (std::log(val) - std::log(1 - val));
}

I didn't know about the Platt scaling, thank you. This library has so many hidden gems!

arrufat on 10 Sep 2020

😄1

You definitely want to do something to calibrate the probability. Like if you say that P(is_a_cat=True | output_of_classifier=123) = 0.8 then it really should be the case that if you took a whole bunch of instances where the classifier output was 123 that 80% of the images were of cats. So any old scaling into 0/1 does not make something a probability. Platt scaling finds the best parametric sigmoid in the sense that it gets you as close to what I just said as possible for a parametric sigmoid.

There are better non-parametric methods. Like you can use isotonic regression, which is in dlib. I don't have a simple thing in dlib that does this kind of probability calibration though. You need to setup the regression in the right way. I've written such codes several times at work, but just never in dlib. But what you do is you use isotonic regression to regress the model output to 0 for false outputs and 1 for true targets. Then you use the Ramer–Douglas–Peucker algorithm to make the resulting curve reasonably compact. And if you really care a whole lot you can follow that up with a least squares refitting to the original non-parametric curve you got from isotonic regression, but fitting it to the joints the RDP found. That's the gold standard for this kind of thing.

For most people that's all overkill though and Platt scaling is just fine. Hardly anyone needs such fine calibration and those that do have very special domain requirements. In fact, most people just want a number they can put in a display for humans to eyeball.

davisking on 10 Sep 2020

❤1

Wow, how can I ever thank you for this kind of comments? It's a domain I am not very familiar with, so you gave me really good pointers to play with it :)

arrufat on 10 Sep 2020

No problem :)

davisking on 10 Sep 2020

👍1

Thank you very much! I have to learn how to perform Platt's scaling.

liskieero on 10 Sep 2020

Warning: this issue has been inactive for 35 days and will be automatically closed on 2020-10-25 if there is no further activity.

If you are waiting for a response but haven't received one it's possible your question is somehow inappropriate. E.g. it is off topic, you didn't follow the issue submission instructions, or your question is easily answerable by reading the FAQ, dlib's official compilation instructions, dlib's API documentation, or a Google search.