Detr: How to run detr with only single class?

[debparth]

I'm trying to run detr on an image whilst having single class, but it always run into an error? Is it because I need to train on my dataset (I don't want to train it on any dataset cause I think that the models works fine for what I want and easily detects what I want to detect). Any Idea how to run it for detecting only single class?

[lessw2020]

Hi @debparth - it's not a problem to train DETR on a single class. (I have done many single class models with DETR).
You say you 'always hit an error' but what is the error?
For reference, you want to set num_classes =1...internally DETR will change that to 2 b/c it's your class + no object class.
Make sure that the category id for the single class in your Coco json is 0.
Also I'm not clear - you don't want to train your dataset but you think DETR will be able to work anyway?
Is this class you want to detect one of the COCO classes already? If so then you could just run the prebuilt DETR and restrict your results to the class you want in the prediction process.

[debparth]

Thank you for replying, contributor.
Actually, I think that I need to detect "Person" class which is already in coco dataset and I don't think to train DETR on coco dataset of person class cause it's already their and also it detects fine so, I'm not thinking to train it. ( Still want your review that should I train it on person class or not? Will it make DETR model to detect better after training)
And I did set num_classes when I'm downloading the model. I already set num_classes = 1 model = torch.hub.load('facebookresearch/detr', 'detr_resnet50', pretrained=True, num_classes = 1) but still not getting it.
The error I get when I try to change the object detect classes to only one class i.e. "person" CLASSES = [ 'person']
I ran into this error:
Sometimes, this error resolves by itself & sometimes doesn't.

If the above error resolves by itself then this error I get if i run the below code.

Could please tell me how to restrict the result of the class in the prediction process too?

[alcinos]

Hi @debparth

Thank you for your interest in DETR.
It is totally possible to use DETR without retraining for a single class like "person", which is very common in COCO. Note however that if you are expecting to detect a lot of people in each image, it might be better to finetune the model on your dataset. See #9 for tips on how to do that.

If you simply want to run inference without retraining, you should NOT change the number of classes when you are loading the model from hub. Instead, you should load it as usual, run the inference as described in our Colab, then ignore (filter out) the predictions that have a class label different than the one you're interested in. The "person" class in COCO has label 1, so you can remove any prediction which doesn't have class 1.

Best of luck.

[debparth]

So, your saying that I need to create my dataset where there are a lot of peoples in single image and thus feed it to DETR. (Am I right!?)

After removing num_classes from the model line and changing this in plot_results function
if text.split(":")[0] == 'person': ax.add_patch(plt.Rectangle((xmin, ymin), xmax - xmin, ymax - ymin, fill=False, color=c, linewidth=3)) ax.text(xmin, ymin, text, fontsize=6, bbox=dict(facecolor='yellow', alpha=0.5))

and now I've got the below output:

Can you please tell me how can I make it better like to detect most of the person in the image?

[fmassa]

@debparth Hi,

Note that DETR currently struggles detecting small objects, so it might not see or be able to accurately localize many of the small instances in your image.

If you increase the image size that is feed to DETR, it will be able to detect more instances of the people in the image.

[debparth]

@fmassa Thank you for your response. I resize the image and it work a bit. Did you try to load a video in DETR!? If you did then can share that code!?

[m3at]

@debparth if you're simply wondering how to repeat your detection over every frame in a video, then save the result as a video, I suggest using imageio (see example), it's a neat library.

If you're asking how to do the detection itself over time (a video), it's a bit out of scope for a github issue but to give you some pointers:

• Without changing the model, you could simply use the IoU of boxes on successive frames, and maybe refine that with Kalman filters. For example, take a look at: https://github.com/bochinski/iou-tracker
• By changing the model, taking multiple frame as input and using attention over time could be a fruitful approach. It's an open area of research 🙂