Detr: Do the model works well in transfer learning?

[riven314]

Thanks for the amazing work!

I noticed the training time for DETR is 3 days with multi GPUs. I believe this setting is too hard to achieve for most end users.

I would like to know in your study did you try transfer learning in DETR? if so, would you provide related module on that?

[lessw2020]

Hi @riven314 -
if it helps - there's discussion and tips on training a custom dataset in this issue #9 if you haven't reviewed that already.
You can either train from scratch or fine tune the head.
As to whether it works well, I suspect a lot will depend on the dataset, size of detected objects etc. In my case, I don't have visualizations setup yet but training loss is decreasing nicely so it's learning at least and will know more later today.
Also regarding gpus - I'm running on a single V100..so unless you are re-training a huge dataset like COCO from scratch you shouldn't need the level of hardware they used in the paper. Main issue is with batch size 2, it's going to go a lot slower relatively.
Hope this info is helpful for now.

[raviv]

@lessw2020 Can you share your loss behavior so far?
I'm also trying to transfer DETR to my domain, which is very different from coco.
You can see below my progress so far, running on an Titan RTX, batch of 12, image size is 1280x720.
I do visualize detections and there are signs of intelligence :) but the loss acts weird.

EDIT: moved this to issue #9

[riven314]

@lessw2020 thanks for the quick address! it helps!
I will close this given it is redundant

[alcinos]

@raviv What loss is this exactly? I'd recommend plotting the loss_bbox, loss_giou and loss_ce individually, it doesn't really help to look and an aggregate.

[raviv]

@alcinos